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ABSTRACT 


The  Asymmetrical  Software  Kit  (ASK)  is  a  software  package  built  for  U.S  Army 
Special  Operations  Command  (USASOC).  It  is  designed  to  greatly  expand  and  digitize 
the  Intelligence  Preparation  of  the  Battlefield  (IPB)  process  for  Special  Forces  units.  The 
purpose  of  this  Thesis  is  to  thoroughly  evaluate  the  Tactical  user’s  acceptance  of  this 
technological  innovation.  Technology  Acceptance  Model,  which  psychometrically 
measures  users’  perceptions  of  ease-of-use  and  utility  to  predict  their  intention  to  use  the 
software,  was  applied  in  this  analysis.  The  test  population  for  this  user  acceptance  survey 
is  the  Tactical  (Group  and  below)  level  user  of  the  ASK.  These  are  the  Special  Forces 
Intel  Sergeants  (18Fs)  on  the  Special  Forces  A-Teams  (ODAs),  and  the  Military 
Intelligence  personnel  at  the  Battalion  and  Group  S2  (Staff  Intelligence)  sections. 
Respondents  completed  an  anonymous,  online  survey  on  their  impressions  of  the  ASK. 
The  questions  were  focused  on  system  usability  and  user  acceptance  in  a  military  setting. 
Overall,  the  models  used  in  this  study  showed  an  acceptable  level  of  fit  with  the  Tactical 
end-user’s  usability  and  acceptance  assessments  and  exhibited  satisfactory  explanatory 
power.  Users  showed  marked  trends  in  response  to  questions  concerning  training, 
command  involvement,  and  system  availability.  Qualitative  input  included  a  number  of 
responses  about  the  idiosyncrasies  of  certain  programs,  and  the  lack  of  high  speed 
computers  to  run  complex  GIS  queries.  The  findings  from  this  study  should  provide 
some  valuable  insights  to  Program  Managers  about  systems  evaluation,  and  clarify  how 
USASOC  can  design  full  spectrum  software  fielding  to  foster  technology  acceptance  and 
use  at  the  Tactical  level. 
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I. 


INTRODUCTION 


A.  BACKGROUND  AND  MOTIVATION 

The  Asymmetrical  Software  Kit  (ASK)  was  designed  using  commercial  off-the- 
shelf  (COTS)  software  and  hardware  to  aid  the  collection,  processing,  and  dissemination 
of  Military  Intelligence  in  Army  Special  Operations  units.  The  changing  nature  of 
warfare  necessitated  a  newer,  more  agile  analytic  system  to  face  newer,  more  agile  foes. 
The  Global  War  on  Terror  (GWOT)  has  highlighted  the  necessity  for  replacing  legacy 
Intel  systems  like  the  All  Source  Analysis  System  (ASAS)  and  its  younger  sibling,  the 
ASAS-Light.  These  systems,  designed  to  operate  in  the  Corps  and  Division  Analysis  and 
Control  Elements  (ACE),  were  left  over  from  the  Cold  War.  According  to  Army  FM  34- 
25-3,  the  ASAS  “is  a  "linchpin"  system  in  forming  a  seamless  intelligence  architecture 
between  and  across  echelons.  They  form  a  system  that  supports  commanders  from 
tactical  through  strategic  levels  anywhere  across  the  range  of  military  operations.”1 


Figure  1 .  Army  Intel  Architecture  and  ASAS 


1  Army  FM  34-25-3,  “All-Source  Analysis  System  And  The  Analysis  And  Control  Element,”  03  Oct 
1995. 
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Engineered  to  process  information  on  Warsaw  Pact  threats  and  push  finished 
intelligence  from  Corps  and  Division  analytic  elements  down  to  Brigade  and  below,  they 
were  wholly  unsuited  to  the  nature  of  the  asymmetric  and  complicated  threat  encountered 
by  Special  Operations  Forces  in  the  GWOT.  Most  Intelligence  requirements  and 
procedures  for  SOF  and  conventional  forces  are  the  same.  Yet,  given  the  uniqueness  of 
SOF  missions  and  roles,  there  are  characteristics  of  intelligence  support  that  require 
unique  systems  and  approaches.  According  to  Army  FM  3-05.102,  Army  Special 
Operations  Forces  Intelligence  these  are — 

•  Level  of  detail.  The  intricate  planning  and  decentralized  execution  of  SOF 
missions  require  intelligence  support  that  is  frequently  more  detailed  or  specific  than  that 
needed  in  conventional  operations.  ARSOF  often  require  intelligence  to  avoid  enemy 
forces,  regardless  of  size  or  composition,  as  opposed  to  information  that  would  allow 
conventional  forces  to  directly  engage  the  enemy. 

•  Timeliness  and  accuracy  of  information.  Timely  and  accurate  intelligence  and 
unprocessed  combat  information  can  be  a  significant  factor  in  operational  success.  Given 
the  high  risk  factor  or  criticality  of  many  ARSOF  missions,  uncertainty  about  the  threat 
or  environment  reduces  the  chances  for  success  more  so  than  in  the  case  of  large-scale 
conventional  operations.  Equal  with  the  need  for  accurate  information  is  the  need  for 
timely  information.  Often  the  success  of  an  ARSOF  mission  hinges  upon  having  a  key  bit 
of  knowledge  at  a  specific  time  or  event. 

•  Operational  element  and  analyst  interface.  Given  the  sensitivity  and  critical 
need  for  very  detailed  information,  especially  in  the  context  of  DA  operations, 
intelligence  personnel  often  need  to  work  directly  with  the  forces  that  will  execute  a 
mission.  Frequently  both  analysts  and  collectors  will  work  hand  in  hand  with  operational 
planners  and  even  with  those  executing  the  actual  mission  during  planning,  rehearsal, 
execution,  and  post-operational  phases  of  a  mission.  Because  ARSOF  units  often  use 
forward  staging  bases,  assigned  intelligence  personnel  must  be  familiar  with  the  conduct 
of  split-based  operations.  This  allows  intelligence  personnel  to  deploy  forward 
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intelligence  “packages”  and  to  maintain  connectivity  with  the  Intelligence  and  Electronic 
Warfare  (IEW)  system  through  communications  with  their  higher  HQ.2 

For  SOF  in  an  asymmetric  fight,  the  legacy  systems  like  ASAS  failed  because 
they  are  not  able  to  meet  the  unique  Intelligence  needs  critical  to  SOF  mission  success. 
The  Asymmetrical  Software  Kit  (ASK)  was  specifically  commissioned  to  confront  this 
change  in  the  Military  Intelligence  paradigm  that  Army  Special  Operators  were 
confronting  in  the  GWOT. 

To  meet  the  needs  of  the  Special  Forces  Intelligence  Sergeant,  or  18F  on  the 
Special  Forced  Operational  Detachment  Alpha  (ODA),  the  ASK  was  originally 
developed  by  cobbling  together  3  COTS  software  programs  -  Analyst’s  Notebook 
(ANB),  Orion  Magic  (OM),  and  ArcGIS.  These  SOF  peculiar  Intelligence  needs  were 
calculated  as  being  able  to  mine  data  (OM),  analyze  networks  (ANB),  and  analyze  and 
display  data  geospatially.  In  2002,  nothing  existed  in  the  military  inventory.  The  Special 
Forces  Intelligence  Sergeant  Course  (SFISC)  committee  at  the  John  F.  Kennedy  Special 
Warfare  Center  and  School  (SWCS)  built  their  own  package  and  insinuated  it  into  the 
curriculum  of  the  SFISC.  The  initial  successes  with  the  system  and  the  analysis  the  18F’s 
performed  as  part  of  their  capstone  exercise  in  the  course  began  to  be  showcased  to 
visiting  dignitaries  and  General  Officers.  The  software  package  quickly  became  fast 
tracked  for  acquisition  and  purchase  by  United  States  Army  Special  Operations 
Command  (USASOC).  In  2004,  fielding  to  individual  Special  Forces,  Civil  Affairs,  and 
other  SOF  units  began.3 

Six  years  into  the  GWOT,  and  three  years  since  the  ASK’s  initial  fielding, 
evidence  from  the  battlefield  suggests  that  ASK  may  not  be  all  that  it  forecast  to  be.  In 
light  of  this  evidence  and  the  resulting  concerns,  it  is  appropriate  to  evaluate  the  ASK 
software  package  by  focusing  on  those  US  military  personal  who  that  have  used  in  on  the 


2  Army  FM  3-05.102,  “Army  Special  Operations  Forces  Intelligence,”  July  2001,  pp.  2-5. 

3  MSG  Matt  Ames  and  CW3  Hendry  during  01  May  2007  interview  with  author  at  Ft.  Bragg,  NC. 
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battlefield.  The  most  important  question  before  us  is  simple.  Is  ASK  really  working  for 
American  soldiers,  and  can  we  measure  and  quantify  this  through  perceived  ease  of  use 
and  perceived  utility? 

To  date,  most  Information  Technology  (IT)  research  has  concentrated  on  user 
acceptance  factors  in  a  business  environment.  In  a  market  economy,  there  are  a  myriad 
of  factors  that  encourage  groups/  users  to  accept  innovations.  In  a  government 
economy,  those  factors  differ  greatly.  According  to  Chau  and  Hu4  and  Lin5,  the 
proliferation  of  innovative  technologies  that  support  individuals  performing  highly 
specialized  tasks  (like  Intelligence  Analysis)  in  the  government  sector  is  one  area  that 
needs  additional  research  into  technology  acceptance  decision-making. 

Investigations  of  technology  usability  and  acceptance  among  tactical  level  (the 
soldiers  on  the  pointy  end  of  the  spear)  users  have  received  precious  little  attention  in 
previous  research.  In  response,  this  project  aims  to  examine  user  acceptance  and 
usability  of  the  ASK,  across  tactical  Army  SOF  units.  This  study  describes  the  design 
and  implementation  of  a  user-centric  evaluation  targeting  system  usability  and  user 
acceptance. 

B  PROBLEM  STATEMENT 

The  ASK,  fielded  at  the  cost  of  millions  of  dollars,  was  designed  to  seamlessly 
integrate  intelligence  up,  down,  and  across  the  Tactical  levels  of  command.  Anecdotally, 
the  system  has  had  major  problems  with  full  adoption.  Users  routinely  leave  the  software 
and  hardware  in  the  rear  when  they  deploy,  or,  at  best,  only  deploy  and  use  certain  pieces 
of  the  system  in  the  capacity  they  were  designed  for,  leaving  many  powerful  tools 
unused.6  Why? 

4  Chau,  P.Y.K.  and  Hu,  P.J.-H.  “Examining  a  model  of  information  technology  acceptance  by 
individual  professionals:  An  exploratory  study,”  Journal  of  Management  Information  Systems,  1 8(4),  2002, 
pp.  191-229. 

5  Chienting  Lin,  Examining  Technology  Usability  and  Acceptance  in  Digital  Government:  A  Case 
Study  in  Law  Enforcement,  Doctoral  Dissertation,  University  of  Arizona,  2004. 

6  These  scenarios  came  to  light  during  the  authors  experience  as  an  Intelligence  Officer  at  3rd  Special 
Forces  Group  from  2004-2006,  and  during  numerous  research  interviews  conducted  at  Ft.  Bragg  in  May 
2007. 
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C  RESEARCH  QUESTIONS 


Evidence  suggests  the  Asymmetrical  Software  Kit  is  so  complex,  and  the  train-up 
is  so  daunting,  that  (as  predicted  by  the  Technology  Acceptance  Model),  tactical  users 
perceived  utility,  acceptance,  and  use  of  the  system  is  low.  If  this  is  demonstrably  true, 
how  can  we  improve  the  software  product  to  increase  perceived  value  and  ease  of  use 
thereby  leading  to  a  high  level  of  user  acceptance  and  use? 

D  INVESTIGATIVE  QUESTIONS 

1 .  Is  ASK  being  accepted  and  used  by  tactical  level  producers  and  consumers  of 
intelligence  in  USASFC  (as  measured  by  their  survey  responses)? 

2.  With  regards  to  ease  of  use,  what,  according  to  the  tactical  level  users,  are  the 
problems  endemic  with  the  ASK?  Do  they  relate  to  ease  of  use,  social  issues,  or  system 
availability? 

3.  How  can  we  increase  user  acceptance  and  use  of  these  types  of  software 
systems?? 

a.  What  is  the  impact  of  increased  training  on  user  acceptance  and  use  (1 8F 
Program  of  Instruction  versus  New  Equipment  Training)? 

b.  Are  there  other  alternatives  (COTS/GOTS  software)  that  contain  attributes 
that  will  increase  perceived  value  and  ease  of  use?  What  are  those  attributes 
independent  of  a  particular  alternative? 

c.  How  can  we  better  plan  for  and  manage  technology  acquisition  to  increase 
acceptance  and  use? 

E  RESEARCH  FOCUS 

There  are  many  different  types  of  collaborative  intelligence  software  that  is  being 
used  in  both  the  private  (financial  analysis)  and  government  (national  intelligence 
agencies)  sectors.  The  focus  of  this  research  will  be  a  thorough  analysis  of  the  factors 
impacting  user  acceptance  of  a  collaborative  intelligence  software  suite  currently  fielded 
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by  Army  Special  Operations  units  at  the  Tactical  level.  The  results  will  be  the  statistical 
output  derived  from  a  proven  theoretical  model  (Technology  Acceptance  Model)  used  to 
predict  future  usage  of  new  Information  Technology  (IT)  and  specific  user-input  for 
improvements  on  the  current  and  future  systems  the  service  will  field. 

F  METHODOLOGY 

To  answer  the  key  research  questions,  a  case  study  will  be  performed  to  identify 
specific  factors  that  impact  user  acceptance  and  usage  of  the  ASK.  This  case  study  will 
be  based  on  a  comprehensive  literature  review  and  interviews  with  tactical  level  users  at 
Ft.  Bragg,  NC.  These  findings  will  be  used  to  construct  the  survey  instrument  used 
during  the  case  study.  A  thorough  statistical  analysis  of  the  survey  data  and  user 
suggestions  will  be  used  to  make  suggestions  for  improvements  to  the  current  system, 
and  generate  requirements  for  fielding  of  future  systems. 

G  SCOPE 

The  scope  of  this  research  effort  will  be  to  explore  the  numerous  factors 
impacting  acceptance  and  use  of  the  Asymmetrical  Software  Kit  among  the  tactical  level 
users.  To  accomplish  this,  the  research  will  review  existing  literature  and  studies  to 
identify  factors  affecting  participation  in  other  forms  of  collaborative  intelligence 
software.  The  goal  of  this  will  be  to  identify  the  key  elements  of  gaining  user  “buy  in” 
and  participation  in  collaborative  intelligence  programs.  The  results  will  potentially  be 
used  to  help  modify  and  manage  the  existing  intelligence  systems,  and  assist  in  the 
design,  implementation,  and  fielding  of  future  software  packages. 

H  LIMITATIONS 

The  limitations  of  this  research  include  the  small  sample  population  (+/-300)  of 
ASK  users  at  the  tactical  level  in  USASOC  units.  This  is  due  to  the  nature  of  the  case 
study  itself.  Outside  factors  have  a  limiting  factor  on  respondents.  USASOC  units  are 
currently  heavily  engaged  in  the  GWOT,  and  are  deployed  across  the  globe,  making 
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contact  with  an  already  small  population  even  more  difficult.  In  addition,  this  project  is 
only  studying  the  Asymmetrical  Software  Kit  at  the  tactical  level.  As  a  result  of  this,  the 
results  of  this  study  may  not  be  applicable  beyond  the  USASOC  community  or  to  IT 
acceptance  theories  in  general. 


7 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


8 


II.  LITERATURE  REVIEW 


A.  OVERVIEW 

This  thesis  research  attempts  to  identify  the  factors  affecting  user  acceptance  of 
the  Asymmetrical  Software  Kit  (ASK)  based  on  the  theories  of  the  Technology 
Acceptance  Model  (TAM).  The  scope  of  this  review  represents  the  learned  opinions  of 
numerous  experts  and  academics  from  different  books  and  journals  pertaining  to 
technology  acceptance  and  use  of  Information  Technologies  (IT).  The  information  in  this 
literature  review  will  attempt  to  explain  the  components  and  function  of  the  ASK  as  well 
as  the  pertinent  factors  that  affect  technology  acceptance  and  adoption. 

B.  THE  ASYMMETRICAL  SOFTWARE  KIT 

The  ASK  consists  of  three  COTS  software  components:  ArcGIS,  Analyst’s 
Notebook,  and  Orion  Magic.  Each  component  serves  as  an  integral  piece  of  the 
Intelligence  analysis  that  tactical  users  do  on  a  continuous  basis.  The  amount  of  data  the 
average  1 8F  takes  in  has  increased  exponentially  in  the  information  age.  These  software 
tools  allow  the  SF  Intelligence  Sergeant  and  the  Military  Intelligence  analysts  he  interacts 
with  at  the  Special  Forces  Battalion  and  Group  levels  to  quickly  store,  process,  analyze, 
and  visualize  unprecedented  amounts  of  information.  The  three  components  each  work 
in  their  own  way  to  facilitate  this  process. 

Data  collection  and  organization  is  the  job  of  SRA  Incorporated’s  Orion  Magic 
(OM)  software.  OM  is  a  “single  framework  for  data  collection,  mining,  and  distribution. 
OM  is  a  suite  of  tools  developed  to  support  investigative  processes,  from  data  collection 
to  statistical  analyses,  to  results  visualization.7  Users  can  logically  organize  incoming 
data  in  a  hierarchical  format  of  cabinets,  outlines,  and  notecards  to  speed  up  subsequent 
data  mining  efforts.  OM  is  also  equipped  with  a  powerful  search  engine  to  search  the 
data  analysts  have  organized.  Their  “Matrix  Search”  feature  “is  a  series  of  vertical 


7  Orion  Magic  website,  http://www.orionmagic.com/overview.htm,  accessed  19  September  2007. 
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column  cells  and  horizontal  row  cells,  each  containing  one  or  more  keywords,  (or 
concepts)  that  can  be  applied  to  the  web,  documents  or  databases  and  return  results  for 
not  only  each  cell  but  also  the  intersection  of  those  cells.”8  Searching  and  displaying 
intersecting  data  in  a  format  like  this  allows  analysts  to  quickly  visualize  a  few, 
significant  intersecting  data  points  in  a  mass  of  information. 
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Figure  2.  File  Cabinets  and  Outlines  in  Orion  Magic 


8  Orion  Magic  website,  http://www.orionmagic.com/search.htm,  accessed  19  September  2007. 

9  Image  downloaded  from  Orion  Magic  website,  http://www.orionmagic.com/collect.htm,  on  19 
September  2007. 
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Figure  3.  Matrix  Search  results  in  OM 


Geo-visualization  and  analysis  of  data  is  another  critical  task  for  the  SF 
Intelligence  Sergeant  and  the  MI  analysts  that  support  him.  This  is  accomplished  using  a 
Geospatial  Information  System  (GIS).  “A  Geographic  Information  System  (GIS)  is  a 
system  for  the  management,  analysis,  and  display  of  geographic  information.  Geographic 
information  is  represented  by  a  series  of  geographic  databases  that  model  geography 
using  simple,  generic  data  structures.” I  1 1  GIS  is  also  defined  as  “an  integrated  system  of 
computer  hardware,  software,  and  trained  personnel  linking  topographic,  demographic, 
utility,  facility,  image  and  other  resource  data  that  is  geographically  referenced.”  12 

ArcGIS  is  a  product  of  ESRI,  a  leading  software  company  in  the  GIS  business.  In 
the  ASK,  ArcGIS  includes:  ArcReader,  which  allows  the  user  to  view  and  query  maps 
created  with  the  other  Arc  products;  ArcView,  which  allows  users  to  view  spatial  data, 
create  maps,  and  perform  basic  spatial  analysis;  ArcEditor  which  includes  all  the 

I  ^linage  downloaded  from  Orion  Magic  website,  http://www.orionmagic.com/search.htm,  on  19  Sep 
2007. 

I I  ESRI  white  paper,  “What  is  ArcGIS?,”  2004,  accessed  on  21  Aug  2007  at 
http://downloads.esri.com/support/documentation/ao_/698What_is_ArcGIS.pdf. 

12  NASA  -  Langley  GIS  Research  Team,  definition  found  on  NASA  website  17  Sep  2007  at  http://gis- 
www.larc.nasa.gov/qat/gisdefmition.html 
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functionality  of  ArcView,  includes  more  advanced  tools  for  manipulation  of  shapefiles 
and  geodatabases;  or  Arclnfo  the  most  advanced  version  of  ArcGIS,  which  includes 
added  capabilities  for  data  manipulation,  editing,  and  analysis.  To  increase  the 
functionality  of  ArcGIS,  special  Military  “Extensions”  have  been  created  to 
accommodate  military  operational  symbols,  military  grid  referencing  systems  (MGRS) 
and  digital  terrain  elevation  data  (DTED).13 


Figure  4.  Data  table  in  MS  Excel  format 


13  ESRI  website,  http://www.esri.com/software/arcgis/extensions/militaryanalyst/index.html.  accessed 
20  September  2007 
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Figure  5.  Data  graphically  displayed  and  overlaid  on  mapping 


Analyst’s  Notebook  (ANB)  is  social  network  analysis  software  from  the  12 
Corporation.  It  allows  analysts  to  process,  analyze,  and  graphically  display  immense 
amounts  of  data  on  networks.  The  Analyst's  Notebook  also  enables  analysts  to  prepare 
and  share  link-analysis  charts.  Figure  6  shows  a  sample  link  analysis  chart  (finished 
intelligence)  from  ANB.  SOF  Intelligence  analysts  use  ANB  to  identify  links  between 
known  or  suspected  terrorists,  their  activities,  phone  numbers,  locations,  and  their 
associations  with  other  persons,  events,  or  groups.  With  ANB,  data  trends  and  linkages 
can  be  visually  displayed,  interpreted,  and  subsequently  analyzed. 


14  Images  downloaded  from  ESRI  Website, 
http://www.esri.com/software/arcgis/about/desktop_gis.html  on  1 8  Sep  2007. 
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Figure  6.  Sample  Link  Analysis  Chart 


C.  TECHNOLOGY  ACCEPTANCE  THEORIES 

In  the  following  sections,  the  preeminent  theories  on  technology  acceptance  and 
adoption  will  be  reviewed.  Classical  Diffusion  of  Innovation  (DOI)  theory  and  the 
leading  intention-based  theories  that  provide  theoretical  underpinnings  of  this  study  will 
be  the  main  focus. 

D.  DIFFUSION  OF  INNOVATION 

The  model  of  “diffusion  of  innovation”  was  proposed  by  Everett  Rogers  in  1962. 
It  was  originally  used  by  rural  sociologists  to  study  the  diffusion  of  agricultural 


15  Image  downloaded  from  12  corporate  website, 
http://www.i2inc.com/Products/Analysts_Notebook/,  accessed  19  Sept.  2007. 


14 


technologies  in  social  systems,  but  it  has  stood  the  test  of  time  and  been  very  successfully 
applied  to  information  technology  products.  16 

According  to  Rogers,  innovation  is  “an  idea,  practice,  or  object  perceived  as  new 

by  an  individual  or  other  unit  of  adoption”^  and  diffusion  is  “the  process  by  which  an 
innovation  is  communicated  through  certain  channels  over  time  and  among  the  members 

of  a  social  system.”  1 8  Rogers  used  well-established  theories  in  sociology,  psychology, 
and  communications  to  develop  a  clear  and  succinct  approach  to  explaining  the  diffusion 
of  innovations. 

The  Innovation;  Adoption  Curve 


1 9 

Figure  7.  Innovation  Adoption  Curve 


After  its  conception,  an  innovation  spreads  slowly  at  first  -  usually  through  the 
campaigning  of  change  agents  -  then  picks  up  speed  and  visibility  as  more  and  more 


' E.  Mustonen-Ollila,  &  K.  Lyytinen,  (2003).  Why  Organizations  Adopt  Information  System  Process 
Innovations;  A  Longitudinal  Study  using  Diffusion  of  Innovation  Theory,  Information  Systems  Journal, 
13(3),  pp.  275-297. 

17  Everett  M.  Rogers,  Diffusion  of  Innovation,  New  York,  Free  Press,  2003,  p.  12. 

18  Everett  M.  Rogers,  Diffusion  of  Innovation,  New  York:  Free  Press,  2003,  p.  35. 

19  Image  downloaded  from  Tulane  University  website,  http://www.payson.tulane.edu/research/E- 
Difflnnova/diff-prob.html  accessed  14  September,  2007. 
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people  are  exposed  to  and  adopt  the  innovation.  Eventually  it  reaches  a  saturation  level, 
where  nearly  everyone  who  is  going  to  adopt  the  innovation,  minus  the  hardcore  luddites, 
has  done  so. 

Individuals  within  a  social  system  do  not  all  adopt  an  innovation  at  the  same  time, 
but  rather  in  an  over-time  sequence,  so  that  individuals  can  be  classified  into  adopter 
categories  based  on  when  they  begin  using  an  innovation.  Socioeconomic  and  individual 
personality  characteristics  define  the  individual’s  role  in  the  diffusion  process.  These 
adopter  roles  are:  innovators  (2.5%),  early  adopters  (13.5%),  early  majority  (34%),  late 
majority  (34%),  and  laggards  (16%). 


Figure  8.  Adopter  Categorization  on  the  Basis  of  Innovativeness 


After  the  forward-thinking  change  agents  have  adopted  the  innovation,  they  work 
to  communicate  it  to  others  in  the  society  by  whatever  means  they  believe  appropriate. 
When  the  number  of  early  adopters  reaches  a  “critical  mass”  -  between  5  and  15%  -  the 
process  becomes  self-sustaining,  as  more  and  more  people  talk  about  or  demonstrate  the 


20 Everett  M.  Rogers,  Diffusion  of  Innovation,  New  York,  Free  Press,  2003,  pp.  280-281. 
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innovation  to  each  other.21  Innovation  diffusion  theorists  have  identified  five  perceived 
attributes22  that  explain  why  some  innovations  reach  this  tipping  point  and  others  never 
do.  They  are: 

Relative  Advantage  -  Is  the  innovation  better  than  the  status  quo?  Will  people 
perceive  it  as  better?  If  not,  the  innovation  will  not  spread  quickly,  if  at  all. 

Compatibility  -  How  does  the  innovation  fit  with  people's  past  experiences  and 
present  needs?  If  it  doesn't  fit  both  well,  it  won't  spread  well.  Does  it  require  a  change  in 
existing  values?  If  members  of  the  culture  feel  as  though  they  have  to  become  very 
different  people  to  adopt  the  innovation,  they  will  be  more  resistant  to  it. 

Complexity  -  How  difficult  is  the  innovation  to  understand  and  apply?  The  more 
difficult  the  innovation  is  to  grasp,  the  slower  the  adoption  process. 

Trialability  -  Can  potential  adopters  "try  out"  the  innovation  first,  or  must  they 
commit  to  it  all  at  once?  If  the  latter,  people  will  be  far  more  cautious  about  adopting  it. 

Observability  -  How  visible/tangible  are  the  benefits  of  adoption?  If  people  adopt 
it,  can  the  difference  be  discerned  by  others?  If  not,  the  innovation  will  spread  more 
slowly.23 

E.  INTENTION-BASED  ACCEPTANCE  THEORY 

Technology  Acceptance  Modeling  (TAM)  builds  on  Rogers’  research  on  patterns 
of  diffusion  by  providing  a  reliable  method  for  predicting  user  behavior.  Three  models 
have  evolved  to  successfully  explain  actual  system  use  by  measuring  users’  perceptions 
towards  the  innovation.  These  are  the  Theory  of  Reasoned  Action  (TRA),  Theory  of 
Planned  Behavior  (TPB),  and  Technology  Acceptance  Model  (TAM). 

According  to  Fishbein  and  Ajzen’s  TRA,  people’s  beliefs  influence  their  attitudes, 
which  together  with  subjective  norms  shape  their  behavior  intentions.  This  then  guides 
or  dictates  their  actual  behavior.  In  numerous  studies,  intention  and  actual  behavior  have 

21  Everett  M.  Rogers,  Diffusion  of  Innovation,  New  York,  Free  Press,  2003,  p.  344. 

22  Perception  of  adopter  is  key  in  Technology  Acceptance  Model  (TAM)  discussed  under  intention- 
based  theories.  DOI  is  the  antecedent  to  TAM. 

23  Everett  M.  Rogers,  Diffusion  of  Innovation,  New  York,  Free  Press,  2003,  p.  265-266. 
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been  shown  to  be  correlated  highly.  TRA  has  been  applied  scientifically  to  explain  a 
wide  variety  of  behaviors,  including  consumer  purchasing24,  condom  usage25,  and  breast 
self-examination.26  TRA  forms  the  basis  for  the  Technology  Acceptance  Model 
ultimately  selected  for  use  in  this  study. 


F igure  9 .  Theory  of  Reasoned  Action2  7 


leek  Ajzen’s  Theory  of  Planned  Behavior  (TPB)  posits  that  attitudes  towards 
behavior  and  the  subjective  norm  from  TRA  are  insufficient  to  explain  behavioral 
intentions.  TBP  introduces  yet  another  factor,  Perceived  Behavioral  Control  (PBC)  to  the 
mix.  PBC  measures  of  the  amount  of  control  people  perceive  they  have  over  their  own 
behavior.  As  a  general  rule,  the  more  favorable  the  attitude  toward  behavior  and 
subjective  norm,  and  the  greater  the  perceived  behavioral  control,  the  stronger  the 
person’s  intention  to  perform  the  behavior  in  question  should  be.  Finally,  given  a 
sufficient  degree  of  actual  control  over  the  behavior,  people  are  expected  to  carry  out 
their  intentions  when  the  opportunity  arises.  28 


24B.  H.  Sheppard.,  J.  Hartwick,  J.&  P.  R.Warshaw,  The  theory  of  reasoned  action:  A  meta-analysis  of 
past  research  with  recommendations  for  modifications  and  future  research,  Journal  of  Consumer  Research, 
vol  15,  1988,  p.  325. 

25  Y.  Kashima,  C  Gallois,  &  M.  McCamish,  The  theory  of  reasoned  action  and  cooperative  behaviour: 
it  takes  two  to  use  a  condom,  British  Journal  of  Social  Psychology,  1993  Sep;32  ( Pt  3):  pp.  227-39  . 

26  L.  M.  Lierman,  H.  M.  Young,  D.  Kasprzyk,  &  J.  Q.  Benoliel,  Predicting  breast  self-examination 
using  the  theory  of  reasoned  action,  Nursing  Research,  1990  Mar-Apr;39(2):  pp.  97-101. 

22  Image  downloaded  from  Your  University  website 
http://www.istheorv.vorku.ca/theoryofreasonedaction.htm  on  07  Dec  2007. 

28  I.  Ajzen,  Residual  Effects  of  Past  on  Later  Behavior:  Habituation  and  Reasoned  Action 
Perspectives.  Personality  and  Social  Psychology  Review,  6(2),  2002,  pp.  107-122. 
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Figure  10. 


Theory  of  Planned  Behavior,  from  Environmental  Health  Journal. 


Fred  Davis  proposed  Technology  Acceptance  Model  (TAM)  in  1989  as  a  logical 
extension  of  TRA  and  TPB.  TAM  theorizes  that  the  two  main  beliefs  determining 
attitude  and  behavioral  intention  are  perceived  usefulness  (PU)  and  perceived  ease  of  use 
(PEOU).  USE  is  defined  as  the  degree  to  which  a  user  believes  that  the  system  he  is 
adopting  will  enhance  his  or  her  performance.  PEOU  refers  to  the  degree  to  which  a  user 
believes  that  using  the  new  system  will  be  free  from  physical  and  mental  effort.30  The 
goal  of  TAM  is  to  predict  and  explain  user  acceptance  of  Information  Technology  (IT) 
across  a  wide  range  of  technologies,  organizational  contexts,  and  user  populations. 
Given  a  choice  between  two  systems,  users  will  generally  rate  the  system  they  find  easier 
to  use  as  having  a  higher  PEOU  score.  TAM  has  been  used  successfully  in  numerous 
studies.  Davis,  Bagozzi,  and  Warshaw  were  able  to  successfully  predict  usage  of  a  word¬ 
processing  program  in  a  class  of  graduate  students31,  and  Dillon  and  Morris  had  similar 
results  predicting  usage  of  a  web-browser  among  computer  science  students.32  The  basic 
TAM  model  is  depicted  below. 

20  Image  downloaded  from  Environmental  Health  Journal  at 
http://www.ehjoumal.net/content/4/l/23/figure/Fl  on  28  Sep  2007. 

30Fred  Davis,  Perceived  usefulness,  perceived  ease  of  use,  and  user  acceptance  of  information 
technology.  MIS  Quarterly,  13(3),  1989,  pp.  319-340. 

31  Fred  Davis,  R.  P. Bagozzi,  &  P.  R.  Warshaw,  User  Acceptance  of  Computer  Technology:  A 
Comparison  of  Two  Theoretical  Models,  Management  Science,  35(8),  1989,  pp.  982-1003. 

32  A.  Dillon  &  M.  Morris,  User  Acceptance  of  Information  Technology:  Theories  and  Models,  Annual 
Review  of  Information  Science  and  Technology,  31,  1996,  pp.  3-32. 
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Figure  1 1 .  Technology  Acceptance  Model 


F.  EVALUATION  METHOD  SELECTION 

After  researching  the  three  models  presented  (TRA,  TPB,  and  TAM),  it  was 
determined  that  TAM  would  best  be  able  to  predict  future  usage  of  the  ASK  for  three 
reasons:  (1)  TAM’s  legitimacy  is  firmly  grounded  in  the  literature  of  psycho-social 
sciences,  (2)  TAM  was  specifically  developed  to  explain  and  predict  user  acceptance  of 
technological  innovations  like  the  ASK,  and  (3)  TAM’s  ability  to  predict  system  usage 
with  a  minimum  amount  of  cost  and  difficulty  are  ideal  for  a  study  like  this. 

G.  THE  RESEARCH  MODEL 

To  date,  only  Chienting  Lin,  Paul  Hu,  and  Hsinchun  Chen  have  looked  at  usage  of 
Intelligence  and  Security  Informatics  (ISI)  systems  (specifically  the  COPLINK  system 
used  by  the  Tucson  PD)  based  on  the  Technology  Acceptance  Model.34  The  model  for 
this  study  is  drawn  from  that  work,  and  is  an  attempt  to  further  validate  their  findings  in  a 
similar  (military  versus  police)  environment. 


33M.  Morris  &  A.  Dillon,  How  User  Perceptions  Influence  Software  Use.  IEEE  Software,  14(4),  1997, 
pp.  58-65.- 

34  Paul  Hu,  Chienting  Lin,.&  H.  Chen,  User  acceptance  of  Intelligence  and  Security  Informatics 
technology:  A  study  of  COPLINK,  Journal  of  the  American  Society  for  Information  Science  and 
Technology,  2005;  56(3)  pp.  235-244. 
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Figure  12. 


Asymmetrical  Software  Kit  User  Acceptance  Study  Research  Model 


Research  suggests  that  a  user’s  decision  to  accept  or  refuse  an  ISI  technology  can 
be  explained  by  factor  pertaining  to  the  technological,  social,  and  organizational  contexts. 
By  examining  these  three  factors,  one  can  determine  the  roots  of  some  of  the  barriers  to 
adoption  in  a  community  of  users. 

The  core  principles  of  the  technological  context  are  perceived  usefulness  (PU), 
perceived  ease  of  use  (PEOU).  For  this  study,  PEOU  and  PU  both  determine  attitude, 
and  PEOU  has  a  direct  and  positive  effect  on  PU.  Efficiency  gain  is  an  extremely 
important  component  in  the  Special  Operations  community:  SOF  peculiar  intelligence 
requirements  are,  as  previously  noted  in  Chapter  1,  Level  of  Detail,  Timeliness  and 
accuracy  of  information,  and  Operational  element  and  analyst  interface.  Intelligence 
analysts  at  the  tactical  level  continuously  work  against  the  clock  to  process,  analyze,  and 
disseminate  intelligence  from  an  ever  increasing  torrent  of  raw  information. 
Accordingly,  the  following  hypotheses  are  tested  in  this  study. 
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Perceived  Usefulness 


HI:  The  level  of  efficiency  gain  of  the  ASK  as  perceived  by  the  tactical  users 
will  positively  affect  their  perceived  usefulness  of  the  ASK. 

H2:  The  level  of  job  relevance  of  the  ASK  as  perceived  by  the  tactical  users  will 
positively  affect  their  perceived  usefulness  of  the  ASK. 

H3:  The  level  of  product  quality  of  the  ASK  as  perceived  by  the  tactical  users  will 
positively  affect  their  perceived  usefulness  of  the  ASK. 

H4:  The  level  of  external  data  access  of  the  ASK  as  perceived  by  the  tactical  users 
will  positively  affect  their  perceived  usefulness  of  the  ASK. 

Perceived  Ease  of  Use 

H5:  The  level  of  external  data  connectivity  of  the  ASK  as  perceived  by  the 
tactical  users  will  positively  affect  their  perceived  ease  of  use  of  the  ASK. 

H6:  The  level  of  functionality  of  the  ASK  as  perceived  by  the  tactical  users  will 
positively  affect  their  perceived  ease  of  use  of  the  ASK. 

H7:  The  level  of  leamability  of  the  ASK  as  perceived  by  the  tactical  users  will 
positively  affect  their  perceived  ease  of  use  of  the  ASK. 

Through  system  exposure  in  training  and  daily  job  usage,  users  in  the  research 
population  have  developed  personal  assessments  of,  and  attitudinal  beliefs  about  the 
ASK.  According  to  TAM  and  TPB,  individuals  embracing  a  positive  attitude  towards  an 
innovation  are  more  likely  to  accept  that  technology  than  those  displaying  a  negative 
attitude.  While  others  have  questioned  the  effectiveness  of  attitude  in  mediating  PEOU 
and  PUs  impact  on  Behavioral  Intent35,  like  Hu,  Lin,  and  Chen,  this  study  has  elected  to 
retain  attitude  because  of  the  considerable  autonomy  afforded  Military  Intelligence 
analysts  and  SF  Intelligence  Sergeants. 


35  Fred  Davis,  Richard  Bagozzi,  &  P.  Warshaw,  User  Acceptance  of  Computer  Technology:  A 
Comparison  of  Two  Theoretical  Models,  Management  Science,  35(8),  1989,  pp.  982-1003. 
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In  the  Social  context  of  this  study,  subjective  norm  refers  to  the  user’s  assessment 
or  perception  of  his  significant  referents’  desires  or  opinions  regarding  his  or  her 
acceptance  of  the  ASK.  In  the  military,  this  is  the  user’s  peer  group  and  his  immediate 
chain  of  command.  In  the  military,  as  in  the  law-enforcement  community  Lin  studied, 
soldiers  form  psychological  attachments  to  their  unit  and  to  the  social  system  within  it.36 
This  attachment,  in  the  military  context,  can  be  attributed  to  personal  commitment  to 
public  service,  the  sense  of  community  in  military  units,  and  the  great  amount  to  which 
soldiers  depend  on  each  other  in  an  increasingly  dangerous  environment.  In  this  study, 
we  measured  that  by  looking  at  Peer  Influence  (PEER).  Given  these  factors,  a  military 
user  is  especially  likely  to  consider  the  ASK  useful  and  exhibit  a  strong  intention  to 
accept  it  when  his  fellow  analysts  and  his  leaders  are  in  favor  of  adopting  the  innovation. 
Since  there  is  a  hierarchy  in  the  profession,  a  military  user  is  also  likely  to  take  into 
consideration  the  wants  and  needs  of  his  supervisors.  This  is  measured  in  the  Chain  of 
Command  (COC)  portion  of  the  survey. 

Subjective  Norm 

H8:  The  level  of  peer  influence  (PEER)  as  perceived  by  the  tactical  users  will 
positively  affect  the  intensity  of  their  perceived  subjective  norm  towards  accepting  the  ASK. 

H9:  The  level  of  command  (COC)  involvement  as  perceived  by  the  tactical  users  will 
positively  affect  the  intensity  of  their  perceived  subjective  norm  towards  accepting  the  ASK. 

System  availability  (AV),  Training  (TNG),  and  Support  (SPT)  are  key 
components  of  the  organizational  context.  System  availability  refers  to  the  users 
perception  of  the  availability  of  equipment  necessary  for  accessing  and  using  the  ASK. 
Resource  availability  is  a  fundamental  construct  of  perceived  behavioral  control,  a 
component  of  Ajzen’s  TPB.  In  Determinants  of  Perceived  Use,  Venkatesh  specifically 
notes  the  effect  of  availability  of  resources  and  opportunities  on  user  acceptance.37  In 


36  Chienting  Lin,  Examining  Technology  Usability  and  Acceptance  in  Digital  Government:  A  Case 
Study  in  Law  Enforcement,  University  of  Arizona,  2004,  p.75. 

37  V.  Venkatesh,  Determinants  of  Perceived  Ease  of  Use:  Integrating  Control,  Intrinsic  Motivation, 
and  Emotion  into  the  Technology  Acceptance  Model,  Information  Systems  Research,  11(4),  p.  342-365.- 
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addition  to  the  previous  research,  the  subjects  of  system  speed,  technical  support,  and 
computer  availability  were  prevalent  themes  in  pre-study  interviews  conducted  with  users 
at  Ft.  Bragg.  There  is  also  a  large  difference  in  the  training  that  SOF  and  MI  MOS 
soldiers  receive  on  the  system.  In  light  of  that  information,  the  study  tested  the  following 
hypotheses. 

Behavioral  Intent 

H10:  The  level  of  system  availability  as  perceived  by  the  tactical  users  will  positively 
affect  the  intensity  of  their  behavioral  intention  towards  accepting  the  ASK. 

HI  1:  The  quality  of  training  as  perceived  by  the  tactical  users  will  positively  affect 
the  intensity  of  their  behavioral  intention  towards  accepting  the  ASK. 

HI 2:  The  level  of  support  as  perceived  by  the  tactical  users  will  positively  affect  the 
intensity  of  their  behavioral  intention  towards  accepting  the  ASK. 


24 


III.  METHODOLOGY 


A.  OVERVIEW 

This  chapter  describes  the  overall  methodology  of  this  research  project.  The 
design  of  the  qualitative  study  will  be  covered  in  depth  to  illustrate  the  processes  and 
procedures  used  to  perform  the  case  study  in  this  research  project. 

B.  TAM  IMPLEMENTATION 

The  following  steps  were  derived  from  a  comprehensive  reading  of  previous 
research:  (1)  identify  an  appropriate  study  group,  (2)  develop  /design  the  survey 
instrument,  (3)  gather  data,  (4)  analyze  the  data,  and  (5)  draw  the  appropriate 
conclusions.  The  following  is  a  discussion  of  how  these  steps  were  implemented. 

C.  IDENTIFY  STUDY  GROUP  PARTICIPANTS 

The  population  of  the  study  consists  entirely  of  Military  Intelligence  personnel, 
Special  Forces  Intelligence  Sergeants,  and  Civil  Affairs  Soldiers  assigned  to  United 
States  Army  Special  Operations  Command  (USASOC).  They  represent  several  different 
Special  Forces  Groups,  the  JFK  Special  Warfare  Center  and  School,  the  96th  Civil  Affairs 
Brigade,  and  soldiers  in  various  subordinate  headquarters  units.  These  soldiers  all  have 
experience  with  the  Asymmetrical  Software  Kit,  and  use  it,  or  components  of  it,  in  the 
course  of  their  analyst  duties.  They  represent  a  broad  range  of  ages,  experience,  job 
training,  and  education. 

D.  INSTRUMENT  DEVELOPMENT 

Multiple  techniques  were  used  during  the  instrument  development  phase. 
Candidate  items  for  the  survey  were  identified  through  previous  research.  The  TAM 
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studies  conducted  on  COPLINK  by  Lin,  Hu  and  Chen38  and  previous  research  done  by 
Lin39  provided  a  readily  validated  survey  instrument.  The  survey  questions  were  drawn 
from  this  work,  and  then  modified  to  reflect  a  change  in  communities  (from  police  to 
military)  and  lexicons.  After  these  changes,  two  Soldiers  from  the  Ft.  Bragg  study  group, 
and  one  Special  Forces  officer  at  the  Naval  Postgraduate  School  voluntarily  assessed  the 
validity  of  the  resulting  items  at  face  value.  From  their  input,  a  few  minor  changes  were 
made  to  tailor  the  survey  to  the  community.  All  question  items  (refer  to  Appendix  B, 
survey  questionnaire)  were  measured  using  a  seven-point  Likert  scale.  The  possible 
response  ranged  from  “strongly  agree”  to  “strongly  disagree.”  Multiple  questions  were 
used  to  measure  single  variables  to  compensate  for  how  subjects  respond  to  questions 
with  certain  word  structures.  To  reduce  the  incidence  of  monotonous  responses  on  the 
survey,  the  sequence  of  questions  was  randomized  and  half  of  the  questions  were 
negated. 

E.  DATA  COLLECTION 

The  study  empirically  tested  the  research  model  using  data  collected  from  a  self- 
administered,  online  survey.  The  link  to  the  website  was  distributed  to  over  350 
participants  through  their  chains  of  command.  All  subjects  participated  in  the  study 
voluntarily.  The  website  with  the  survey  was  kept  open  for  eight  weeks,  from  13 
September  2007  to  12  November  2007  to  allow  as  many  subjects,  given  the  current  unit 
operational  tempo  and  competing  deployments,  to  reply  as  possible.  Of  the  Soldiers  that 
it  was  emailed  to,  38  initiated  the  survey  on  the  website.  Of  those  38  hits,  25  produced 
useable  data. 

The  survey  was  designed  to  collect  both  qualitative  and  quantitative  data.  The 
constructs  were  measured  using  a  seven  point  Likert  Scale,  with  one  as  “Strongly 


38  Paul  Hu,  Chienting  Lin,  &  H.  Chen,  User  acceptance  of  Intelligence  and  Security  Informatics 
technology:  A  study  of  COPLINK,  Journal  of  the  American  Society  for  Information  Science  and 
Technology,  2005;  56(3)  p.  235-244. 

39  Chienting  Lin,  Examining  Technology  Usability  and  Acceptance  in  Digital  Government:  A  Case 
Study  in  Law  Enforcement,  University  of  Arizona,  2004. 
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Disagree”  and  seven  as  “Strongly  Agree.”  See  Table  1  below  for  a  sample  question  from 
the  survey  with  the  accompanying  seven  point  Likert  Scale  below  it.  Multiple  questions 
were  used  to  measure  single  variables,  allowing  for  variation  in  how  individuals  respond 
to  varying  word  structures.  The  process  of  determining  which  sets  of  questions  exhibit 
the  smallest  amount  of  variance  for  a  particular  variable  is  called  Reliability  Analysis. 
Reliability  is  calculated  using  a  statistical  procedure  to  measure  Cronbach’s  Alpha.40 
Cronbach  Alpha  ranges  between  0.0  and  1.0,  and  value  of  greater  than  0.7  is  considered 
sufficient  for  social  research. 


12.  Using  the  ASK  woulc 

improve  my  job  performance 

Strongly 
Agree  with 
the 

statement 

Agree 
with  the 

statement 

Slightly 
Agree 
with  the 
statement 

Neither 

Agree 

nor 

Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 
statement 

Disagree 
with  the 

statement 

Strongly 
Disagree  with 
the  statement 

Table  1.  Sample  Survey  Question41 


In  his  initial  research  on  TAM,  Fred  Davis  performed  this  measure  on  each  of  his 
four  variables,  Perceived  Utility  (PU),  Perceived  Ease  of  Use  (PEOU),  Attitude  (ATT) 
towards  using,  and  Behavioral  Intent  to  use  (BI),  to  ascertain  their  reliability.  His  four 
measures  of  PU  achieved  a  reliability  score  of  0.92,  and  PEOU  achieved  0.90.  The  ATT 
scale  and  BI  scale  obtained  reliability  scores  of  0.82  and  0.92  respectively.42  The  same 
analysis  was  repeated  on  the  data  gathered  for  this  study  to  ensure  reliability. 


40L.  Cronbach,  Coefficient  alpha  and  the  internal  structure  of  tests.  Psychometrika,  16(3),  1951,  pp. 
297-334. 

41  V.  Venkatesh  and  F.  Davis,  A  Model  of  Antecedents  of  Perceived  Ease  of  Use:  Development  and 
Test.  Decision  Sciences,  27(3)  1996,  pp.  451-481. 

42  V.  Venkatesh,  and  F.  Davis,  A  Model  of  Antecedents  of  Perceived  Ease  of  Use:  Development  and 
Test.  Decision  Sciences,  27(3)  1996,  pp.  451-481. 


27 


In  addition  to  the  TAM  constructs  investigated  by  Davis,  this  study  also 
investigates  users  perceptions  of  system  Functionality  (FUNC),  available  Tech  Support 
(SPT),  the  importance  of  External  Connectivity  (EXCON),  Efficiency  Gain  (EG)  and  the 
Training  (TNG)  they  have  received  on  the  system.  These  questions  were  modeled  after 
the  studies  on  user  acceptance  of  COPLINK  conducted  by  Lin.43 

To  assess  the  qualitative  aspect  of  the  survey,  three  open  ended  questions  were 
asked  at  the  end  of  the  survey.  Respondents  were  elicited  for  their  input  on  what  part  of 
the  system  worked  well,  what  needed  fixing,  and  their  suggestions  for  future 
improvements.  A  sample  of  their  input  is  included  in  Chapter  IV. 


43  Chienting  Lin,  Examining  Technology  Usability  and  User  Acceptance  In  Digital  Government:  A 
Case  Study  In  Law  Enforcement,  University  of  Arizona,  2004. 
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IV  RESULTS  AND  ANALYSIS 


A.  OVERVIEW 

This  chapter  presents  the  results  of  the  research  study  in  four  distinct  sections: 
Instrument  Validation,  Respondent  Demographics,  Qualitative  Feedback,  and 
Quantitative  Analysis.  The  survey  instrument  was  validated  using  Reliability  Analysis. 
The  subsequent  sections  examine  respondent  demographics;  summarize  some  user¬ 
generated  qualitative  feedback  about  the  system,  and  statistically  analyze  the  survey  data 
to  find  correlations  to  the  model. 

B.  INSTRUMENT  VALIDATION 

Most  of  the  items  in  this  survey  were  drawn  from  previously  validated 
measurements,  but  this  study  re-examines  the  instrument’s  convergent  and  discriminate 
validity.  Using  the  responses  from  the  survey,  the  instrument’s  convergent  validity  was 
examined  using  the  Cronbach’s  alpha  value  for  each  of  the  specified  constructs.  Many 
social  science  professionals,  as  a  rule  of  thumb,  require  a  Cronbach's  alpha  reliability 
score  of  0.70  or  higher.44  Cronbach's  alpha  is  viewed  as  a  measure  of  how  well  the  sum 
score  of  the  selected  items  capture  the  expected  score  for  the  larger  domain,  even  if  that 
domain  is  heterogeneous.  The  results  of  the  reliability  analysis  are  displayed  in  Table  3 
below.  The  Cronbach’s  Alpha  values  ranged  between  .073  and  .095,  exhibiting  more 
than  adequate  reliability. 

To  evaluate  the  instrument’s  discriminate  validity,  principal  component  analysis 
was  performed  using  a  varimax  with  rotation.  Discriminate  validity  is  satisfactory  when 
items  show  low  loading  on  other  constructs.  Each  component  showed  an  eigenvalue 
greater  than  1.0,  the  standard  threshold  for  factor  analysis.  Ideally,  all  the  components 
would  load  on  separate  factors  and  achieve  an  eigenvalue  greater  than  1.0.  This  did  not 

44  Wikipedia  page  on  Cronbach’s  Alpha,  http://en.wikipedia.org/wiki/Cronbach's_alpha,  accessed 
June  18,  2007. 
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occur.  However,  the  purpose  of  this  research  was  to  better  understand  the  level  of 
variance  explained  by  variables  impacting  the  Technological,  Social,  and  Organizational 
constructs  within  the  context  of  the  TAM  and  as  applied  to  the  Asymmetrical  Software 
Kit,  as  opposed  to  improving  the  model.  When  examining  the  variance  explained  by 
factors  influencing  PU  and  PEOU,  both  a  sample  size  of  25  and  higher  than  expected 
correlations  between  independent  variables  may  be  affecting  the  level  of  variance 
explained. 

Linear  regression  in  SPSS  was  used  to  examine  the  relationships  within  the 
Technological,  Organizational,  and  Social  constructs  of  the  TAM  The  hypothesized 
relationships  delineated  in  this  model  were  shown,  by  examining  Beta  Coefficient  and 
Significance,  to  explain  a  significant  portion  of  the  variance  in  all  three  constructs. 

C.  RESPONDENT  DEMOGRAPHICS 

The  overwhelming  majority  of  our  respondents  (97%)  were  males,  over  the  age  of 
30  and  with  12  or  more  years  of  service  in  the  Army.  Most  of  these  were  senior  Non- 
Commissioned  Officers  (NCO’s)  and  Warrant  Officers.  They  were  split  evenly  between 
Military  Intelligence  (MI)  and  Special  Forces  Military  Occupational  Skills  (MOS’s).  On 
a  whole,  they  were  very  well  educated,  with  82.4%  having  some  college,  and  34.2% 
having  a  College  or  Postgraduate  degree.  They  are  familiar  with  computers  and  use  them 
regularly:  94.4%  used  their  personal  computer  at  least  once  a  day,  and  97.3%  used  their 
work  computers  once  a  day  or  more.  A  full  65%  of  the  respondents  indicated  they  had  a 
year  or  more  of  experience  using  the  ASK.  For  system  training,  39.4%  received  their 
training  on  the  ASK  from  the  18F/  Special  Forces  Intelligence  Sergeant  Course  (SFISC), 
a  12  week  course  at  Ft.  Bragg,  NC.  Of  the  remaining  respondents,  7.9%  indicated  they 
had  received  the  two-week  course  from  the  New  Equipment  Training  Team.  Fully  52.7% 
of  the  respondents  had  received  either  no  formal  training,  or  only  received  On  the  Job 
Training  (OJT)  from  their  fellow  analysts  or  supervisors. 

Most  of  the  respondents  interact  with  computers  at  work  and  at  home  on  an 
almost  daily  basis.  Their  mean  score  for  usage  at  home  and  work  were  4.4  and  5.5 
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respectively.  This  indicates  they  use  computers  at  home  at  least  once  a  day,  and  use 
computers  at  work  more  than  three  to  five  times  a  day.  Respondents  were  also  asked  to 
rank  their  usage  of  the  components  in  the  ASK  (Orion  Magic,  Analyst  Notebook,  and 
ArcGIS).  The  table  below  displays  their  responses.  A  preponderance  of  the  respondents 
used  the  software  less  than  once  a  day.  For  Orion  Magic  and  ArcGIS  respectively,  64.8% 
and  52.6%  rarely  or  never  used  the  programs.  The  mean  scores  for  component  usage 
were  low.  ArcGIS  and  Analysts  scored  a  mean  reported  usage  of  2.8  each,  which  equates 
weekly  usage.  Orion  Magic  had  a  mean  reported  usage  of  2.4,  which  is  closer  to  monthly 
usage. 
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Figure  13.  ASK  Component  Usage 


D.  QUALITATIVE  FEEDBACK: 

The  survey  contained  three  questions  at  the  end  requesting  qualitative  feedback 
on  the  ASK.  The  following  are  a  sample  of  their  responses  to  these  questions.  A 
complete  listing  can  be  found  in  Appendix  C. 
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1.  “What  features  of  the  ASK  do  you  like  most?  Why?” 

-  ArcGIS  -  industry  leader  to  allow  for  utilization  of  GIS  information.  We  need 
software  that  allows  us  to  conduct  analysis  of  a  digital  COP.  ANB  -  very  versatile 
application  for  link  analysis.  GeoRover  -  Makes  utilizing  ArcGIS  a  WHOLE  LOT  easier 
and  much  more  efficient. 

-  Orion  Magic  is  a  great  help  to  researching  local  databases.  ARCGIS  has 
practical  application  in  my  job.  Analyst  Notebook  helps  to  visualize  intelligence 
reporting. 

-  Orion  Magic  is  one  of  the  best  search  tools  I  have  worked  with.  The  problem  is 
access  to  all  the  databases 

- 1  like  ArcGIS  for  producing  integrating  data,  performing  analysis  and  producing 
shape  files  and  databases;  however,  simpler  more  intuitive  tools  may  be  more  useful  for 
most  common  tasks  related  to  spatial  visualization  (IE:  Google  Earth,  etc).  Allowing  both 
spatial  capabilities  may  bridge  the  gap  between  analysts  and  operators. 


2.  “What  features  of  the  ASK  do  you  like  the  least?  Why?” 

-  ArcGIS  is  not  easy  to  use,  and  requires  extensive  training.  ArcGIS  only  works 
with  powerful  computers  which  combat  units  do  not  have.  Finally,  no  help  is  provided  to 
the  users. 

-  I  detest  Analyst  Notebook  because  link  charts  become  too  crowded;  charts 
cannot  be  easily  searched  for  key  words;  and  system  does  not  allow  for  true  social 
network  analysis  functions.  I  would  recommend  exploring  and  developing  alternative 
link-analysis  platforms. 

-  ARCmap  has  too  many  features  and  too  many  different  ways  to  do  the  same 
thing.  This  makes  the  program  bigger  and  harder  to  use.  When  my  PC  locks  up  I  can 
loose  hours  of  work. 
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3.  “What  suggestions  do  you  have  for  future  improvements  to  the  ASK 
concept?” 

-  We  need  better  computers,  better  training,  and  Imbedded  support  before  and 
during  deployments. 

-  Bn  S3's,  S2's,  and  Officers  who  will  be  commanders  at  BN  and  Group  level  need 
to  be  familiar  with  ASK  so  they  can  effectively  work  with  Intel  fusion.  It  is  very 
disheartening  to  spend  over  100  man  hours  doing  a  130pg  SODARS  with  photo's,  link 
charts,  and  ARCmap  products  to  have  it  locked  in  the  safe  by  your  S3  CW4  that  you 
submit  it  too  so  it  is  never  accessible/searchable/submitted  to  the  portal,  or  useful  to 
anyone. 

-  The  concept  of  mapping  people,  places  things  and  events  across  time  and  space 
is  solid.  However,  the  process  requires  re-design  to  ensure  accessibility  and  acceptance. 
Recommend  redesigning  simple  and  intuitive  data-centric  systems  based  on  technologies 
which  present  low  barriers  to  entry  and  learning. 

E.  RESULTS  OF  QUANTITATIVE  ANALYSIS 

1.  Previous  Research 

When  Hu,  Lin,  and  Chen  ran  their  study  on  software  acceptance  in  a  Law 
Enforcement  context,  their  findings  suggested  a  “prominent  core  influence  path  from 
efficiency  gain  to  perceived  usefulness  and  then  to  intention  to  accept.”45  Lin  also  noted 
that  “Perceived  usefulness  appears  to  be  the  single  most  acceptance  driver  for  law 
enforcement  officers,  showing  a  highly  prominent  and  significant  direct  effect  on 
behavioral  intention.”46  They  also  found  the  insignificance  of  perceived  ease  of  use  on 
perceived  usefulness  indicated  a  professional  orientation  amongst  officers  that  had  been 


45  Paul  Hu,  Cjienting  Lin,  &  H.  Chen,  User  acceptance  of  Intelligence  and  Security  Informatics 
technology:  A  study  of  COPLINK,  Journal  of  the  American  Society  for  Information  Science  and 
Technology,  2005;  56(3)  p.  242. 

46  Chienting  Lin,  Examining  Technology  Usability  and  Acceptance  in  Digital  Government:  A  Case 
Study  in  Law  Enforcement,  University  of  Arizona,  2004,  p.  88. 
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suggested  by  earlier  research.47  They  also  found  subjective  norm  had  a  significant  effect 
on  acceptance  decision-making,  but  that  normative  beliefs  themselves  did  not  lead 
directly  to  user  acceptance.48 

2  Results  of  this  Study 

In  this  study,  Perceived  Usefulness  exists,  along  with  Perceived  Ease  of  Use  in 
the  Technological  context.  The  mean  value  of  Perceived  Usefulness  was  5.550,  meaning 
users  perceived  the  ASK  to  be  somewhat  useful  software.  The  mean  of  Job  Relevance 
was  5.140  and  Product  Quality  was  5.190.  Efficiency  Gain  (EG)  was  4.933,  indicating 
the  respondents  may  perceive  a  net  loss  of  efficiency  if  they  have  to  spend  an  inordinate 
amount  of  time  manipulating  an  overly  cumbersome  system.  This  theme  was  carried 
over  in  the  qualitative  responses  and  during  the  interviews  at  Ft.  Bragg.  Using  linear 
regression,  a  coefficient  of  determination  (R  )  and  a  regression  coefficient  (Beta)  were 
determined  for  the  model.  For  Perceived  Usefulness,  the  adjusted  R  value  was  .716.  Job 
Relevance  and  Product  Quality  were  the  most  important  factors.  Efficiency  Gain  only 
had  a  small  effect  on  Perceived  Usefulness,  and  External  Connectivity  had  little  to  no 
bearing  on  Perceived  Usefulness. 

These  results  show  that  Hypotheses  Two  (The  level  of  job  relevance  of  the  ASK 
as  perceived  by  the  tactical  users  will  positively  affect  their  perceived  usefulness  of  the 
ASK)  and  Three  (The  level  of  product  quality  of  the  ASK  as  perceived  by  the  tactical 
users  will  positively  affect  their  perceived  usefulness  of  the  ASK.)  were  shown  to  be 
valid.  Hypothesis  One:  (The  level  of  efficiency  gain  of  the  ASK  as  perceived  by  the 
tactical  users  will  positively  affect  their  perceived  usefulness  of  the  ASK.)  and 
Hypothesis  4:  The  level  of  external  data  connectivity  of  the  ASK  as  perceived  by  the 
tactical  users  will  positively  affect  their  perceived  usefulness  of  the  ASK.)  were  found 


47  M.  S.  Larson,  The  Rise  of  Professionalism:  A  Sociological  Analysis,  Berkley,  CA,  University  of 
California  Press,  1977. 

48  Chienting  Lin,  Examining  Technology  Usability  and  Acceptance  in  Digital  Government:  A  Case 
Study  in  Law  Enforcement,  University  of  Arizona,  2004,  p.  89. 
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not  to  correlate  as  strongly.  Statistically,  Job  Relevance  and  Product  Quality  are  the  two 
strongest  determinants  of  Perceived  Usefulness. 


Construct 

Item 

Mean 

STD 

Cronbach’s  a 

Perceived  Utility  (PU) 

PU-1 

5.480 

1.610 

0.958 

PU-2 

5.600 

1.154 

PU-3 

5.560 

1.325 

PU-4 

5.560 

1.416 

Efficiency  Gain  (EG) 

EG-1 

4.880 

1.301 

0.801 

EG-2 

5.040 

1.513 

EG-3 

4.880 

1.452 

Job  Relevance  (JOBR) 

JOBR-1 

4.920 

1.497 

0.821 

JOBR-2 

5.200 

1.554 

JOBR-3 

4.880 

1.536 

JOBR-4 

5.560 

1.157 

Product  Quality 
(PRODQ) 

PRODQ- 1 

5.720 

1.137 

0.821 

PRODQ-2 

5.240 

1.588 

PRODQ-3 

4.400 

1.755 

PRODQ-4 

5.400 

1.527 

Table  2.  Reliability  Analysis  for  Perceived  Usefulness 


The  mean  score  for  Perceived  Ease  of  Use  was  4.690.  The  system  is  not 
perceived  to  be  more  than  moderately  easy  to  use.  For  Functionality  and  Learn  ability, 
the  mean  scores  4.246  and  4.680  respectively.  This  indicates  users  found  the  ASK 
moderately  functional  in  its  layout,  and  rated  their  prospect  of  learning  how  to  use  the 
system  at  somewhat  better  than  average.  Of  the  factors  bearing  on  Perceived  Ease  of 
Use,  Functionality  and  Learn  ability  were  the  most  important  determinants  of  Perceived 
Ease  of  Use,  Functionality  correlated  to  Perceived  Ease  of  Use  at  .841  and  Learn  ability 
correlated  at  .680. 

Hypotheses  Six  (The  level  of  functionality  of  the  ASK  as  perceived  by  the  tactical 
users  will  positively  affect  their  perceived  ease  of  use  of  the  ASK.)  and  Seven  (The  level 
of  learn  ability  of  the  ASK  as  perceived  by  the  tactical  users  will  positively  affect  their 
perceived  ease  of  use  of  the  ASK.)  were  proven  to  have  a  statistically  significant 
correlation.  They  are  the  strongest  determinants  of  Perceived  Ease  of  Use.  Hypothesis 
Five  (The  level  of  external  data  connectivity  of  the  ASK  as  perceived  by  the  tactical  users 
will  positively  affect  their  perceived  ease  of  use  of  the  ASK.)  was  again  proven  to  be 
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uncorrelated  to  both  factors  in  the  Technological  Construct.  External  Connectivity,  while 
important  to  users,  has  no  bearing  on  either  Perceived  Usefulness  or  Perceived  Ease  of 
Use. 


Construct 

Item 

Mean 

STD 

Cronbach’s  a 

Perceived  Ease  of  Use 
(PEOU) 

PEOU-1 

4.080 

1.552 

0.875 

PEOU-2 

4.880 

1.394 

PEOU-3 

4.800 

1.471 

PEOU-4 

5.000 

1.384 

Functionality  (FUNC) 

FUNC-1 

4.480 

1.636 

0.905 

FUNC-2 

4.320 

1.700 

FUNC-3 

4.360 

1.680 

FUNC-4 

3.880 

1.763 

Feam  ability  (FEARN) 

FEARN- 1 

4.760 

1.640 

0.739 

FEARN-2 

4.720 

1.882 

FEARN-3 

4.800 

1.290 

FEARN-4 

4.440 

1.709 

External  Connectivity 
(EXCON) 

EXCON- 1 

5.320 

2.267 

0.759 

EXCON-2 

5.680 

1.651 

EXCON-3 

5.720 

2.051 

EXCON-4 

5.360 

2.215 

Table  3.  Reliability  Analysis  for  Perceived  Ease  of  Use 


Figure  14. 


(##)=  Beta  Coefficient 
Technological  Construct  Regression  Values 
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The  mean  of  Subjective  Norm  was  4.400,  indicating  an  overall  moderate  to  weak 
influence  of  peers  and  leaders  in  influencing  adoption  behavior.  Peer  Influence  mean 
was  4.733,  and  Chain  of  Command  influence  mean  was  5.040.  Users  are  influenced  by 
their  peers  to  a  moderate  extent,  but  not  by  their  Chain  of  Command.  Peer  Influence 
correlates  to  Subjective  Norm  at  .592,  but  Chain  of  Command  influence  correlates  at  - 
.004.  Tactical  level  SOF  users  are  trained  to  work  in  small  groups  and  rely  on  each  for 
their  survival.  It  is  not  surprising  that  soldiers  trained  to  this  extent  would  hold  the 
opinions  of  their  peers  in  higher  regard  than  the  opinions  of  their  supervisors.  It  is  very 
likely  that  command  influence  in  a  SOF  unit  is  not  as  important  a  driver  in  a  SOF  unit  as 
it  would  be  in  a  conventional  Army  unit,  but  this  bears  further  research. 

Hypothesis  Eight  (The  level  of  peer  influence  (PEER)  as  perceived  by  the  tactical 
users  will  positively  affect  the  intensity  of  their  perceived  subjective  norm  towards 
accepting  the  ASK.)  proved  a  statistically  significant  correlation.  Conversely, 
Hypothesis  Nine  (The  level  of  command  (COC)  involvement  as  perceived  by  the  tactical 
users  will  positively  affect  the  intensity  of  their  perceived  subjective  norm  towards 
accepting  the  ASK.)  correlated  negatively.  This  may,  as  mentioned  earlier,  be  a  function 
of  the  way  in  which  Special  Operations  Soldiers  are  trained  to  operate  in  small,  self 
sufficient  units  far  from  the  direct  command  and  control  exercised  by  many  conventional 
military  commands. 


Construct 

Item 

Mean 

STD 

Cronbach’s  a 

Subjective  Norm  (SN) 

SN-1 

5.280 

1.369 

0.733 

SN-2 

5.320 

1.405 

Chain  of  Command 
(COC) 

COC-1 

5.320 

1.405 

0.741 

COC-2 

5.360 

1.150 

COC-3 

5.400 

1.554 

COC-4 

5.280 

1.458 

Peer  (PEER) 

PEER-1 

5.400 

1.290 

0.363 

PEER-2 

4.920 

1.605 

PEER-3 

3.880 

1.877 

Table  4.  Reliability  Analysis  for  Subjective  Norm 
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(##)=  Beta  Coefficient 

Figure  15.  Social  Construct  Regression  Values 

In  the  Organizational  construct,  respondents  displayed  a  Behavioral  Intent  to  use 
mean  of  5.346.  This  indicates  a  slightly  positive  intent  to  use  the  system.  Support  had  a 
mean  score  of  4.040,  indicating  a  neutral  opinion  of  the  support  mechanisms  for  the 
software.  Training  also  scored  a  mean  of  4.550,  which  is  neutral.  However,  when  you 
reanalyze  the  data,  SOF  MOS  soldiers  scored  a  mean  of  4.43  for  Training,  while  MI 
MOS  soldiers  scored  a  mean  of  4.64.  This  demonstrates  a  slight  disparity  in  the 
perception  of  the  training  soldiers  receive.  System  Availability  scored  in  the  negative 
region,  with  a  mean  of  3.160.  Soldiers  clearly  thought  system  availability  and  finding 
enough  computers  appropriate  to  run  the  software  were  problem  facing  their  acceptance 
of  the  system.  These  concerns  were  bom  out  in  the  qualitative  feedback  from  the  survey. 
For  Behavioral  Intent,  Training  and  Support  were  the  most  important  factors.  Training 
correlated  to  Behavioral  Intent  at  .498,  and  Support  correlated  at  .310.  The  adjusted  R- 
squared  value  for  the  regression  analysis  of  the  Organizational  construct  was  .215.  These 
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low  scores  predict  a  likelihood  of  marginal  ASK  use  by  the  respondents.  This  is  bom  out 
in  the  usage  data  and  the  comments  from  the  qualitative  data. 

Hypothesis  Ten  (The  level  of  system  availability  as  perceived  by  the  tactical  users 
will  positively  affect  the  intensity  of  their  behavioral  intention  towards  accepting  the 
ASK.)  did  not  correlate  strongly,  although  respondents  did  express  much  concern  over 
the  systems.  Hypotheses  Eleven  (The  quality  of  training  as  perceived  by  the  tactical 
users  will  positively  affect  the  intensity  of  their  behavioral  intention  towards  accepting 
the  ASK.)  and  Twelve  (The  level  of  support  as  perceived  by  the  tactical  users  will 
positively  affect  the  intensity  of  their  behavioral  intention  towards  accepting  the  ASK.) 
proved  the  strongest  correlations  with  Behavioral  Intent.  Training  and  Support,  in  that 
order,  were  the  most  important  determinants. 


Construct 

Item 

Mean 

STD 

Cronbach’s  a 

Availability  (AV) 

AV-1 

2.800 

1.527 

0.786 

AV-2 

3.480 

1.782 

AV-3 

3.000 

1.607 

AV-4 

3.360 

1.468 

Support  (SPT) 

SPT-1 

4.040 

2.051 

0.864 

SPT-2 

4.120 

1.943 

SPT-3 

3.960 

1.881 

Training  (TNG) 

TNG-1 

4.480 

1.837 

0.753 

TNG-2 

4.280 

1.744 

TNG-3 

5.240 

1.200 

TNG-4 

4.200 

1.658 

Table  5.  Reliability  Analysis  for  Behavioral  Intent 
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System  Availability 
(AV) 


(0.104) 


Support 

(SPT) 


(0.310) 


Adj  R 
0.215 


(##)=  Beta  Coefficient 

Figure  16.  Organizational  Construct  Regression  Values 


F  CONCLUSIONS  AND  DISCUSSION 

Findings  from  this  analysis  have  several  implications  for  research  and  technology 
fielding  practices  in  the  military  context.  The  analysis  suggests  that  Perceived 
Usefulness  appears  to  be  the  most  important  acceptance  driver  for  users  at  the  Tactical 
level.  Job  Relevance  and  Product  Quality  were  the  two  most  significant  drivers  of 
Perceived  Usefulness,  suggesting  that  users  may  anchor  their  acceptance  decision  from  a 
utility  perspective.  This  was  very  similar  to  findings  with  law-enforcement  officers  and 
the  COPLINK  software.49 

Second,  Peer  Influence  seem  to  be  the  strongest  driver  of  Subjective  Norm. 
When  assessing  the  utility  of  the  ASK,  Soldiers  are  likely  to  take  into  account  the 
opinions  of  many  people.  What  is  significant  here  is  that  the  Chain  of  Command  held 

49  Chienting  Lin,  Examining  Technology  Usability  and  Acceptance  in  Digital  Government:  A  Case 
Study  in  Law  Enforcement,  University  of  Arizona,  2004,  p.  88. 
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almost  no  sway  in  a  soldier’s  decision.  This  may  be  due  to  the  command’s  lack  of 
involvement  with,  or  knowledge  about  the  ASK. 

Finally,  while  System  Availability  is  perceived  as  a  problem  among  the 
respondents,  it  is  not  a  significant  driver  of  Behavioral  Intent  to  use.  Respondents  had  a 
lot  to  say  about  the  availability  and  appropriateness  of  their  computers  in  both  pre-survey 
interviews  at  Ft.  Bragg  and  in  the  qualitative  portion  of  the  survey  itself.  Training  and 
technical  Support  for  the  ASK  are,  however,  significant  aspects.  As  noted  before,  there  is 
a  difference  in  perceptions  of  training  amongst  the  respondents,  and  it’s  split  along  MOS 
lines.  Respondents  also  mentioned  a  perceived  lack  of  dedicated  tech  support  for  the 
system,  which  hampered  their  efforts  at  proficiency  during  pre-mission  train  up  cycles. 

Design  and  use  of  collaborative  Intelligence  analysis  software  like  the  ASK  can 
greatly  enhance  the  war  fighting  capabilities  of  SOF  units.  Any  unit  adopting  new 
technology  must  carefully  manage  that  process  to  reap  the  full  benefit  of  the  innovation. 
Driven  by  a  need  to  explore  technology  acceptance  by  Tactical  level  users,  this  study 
designed  and  empirically  tested  a  model  for  explaining  individual  users  acceptance  of  the 
ASK.  Overall,  the  model  showed  a  good  fit  with  the  survey  data  collected,  and  it 
exhibited  satisfactory  explanatory  power  to  most  of  the  acceptance  decisions  observed. 
These  findings  will  help  instructors,  software  designers,  and  leaders  design  systems  and 
fielding  plans  that  foster,  rather  than  stifle  technology  acceptance  by  Tactical  users. 
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V.  RECOMMENDATIONS  AND  CONCLUSIONS 


A.  OVERVIEW 

From  the  beginning,  the  purpose  of  this  research  has  been  to  identify  those  factors 
relating  to  user  acceptance  of  the  Asymmetrical  Software  Kit  (ASK).  This  chapter  will 
summarize  the  research,  examine  the  implications  of  significant  findings,  and  identify 
opportunities  for  continued  research  in  this  field.  The  Technology  Acceptance  Model 
(TAM)  results  for  this  study  are  summarized,  and  some  implications  for  further  TAM 
usage  are  delineated.  Finally,  based  off  all  the  analysis,  recommendations  are  given  for 
continued  training,  fielding,  and  evolution  of  the  ASK  and  the  systems  that  will  succeed 
it. 

With  a  better  understanding  of  the  key  drivers  of  user  acceptance  for  the  tactical- 
level  user,  leaders,  trainers,  and  procurement/futures  professionals  can  quickly  identify 
areas  where  user  acceptance  is  currently  being  hindered,  and  likely  to  be  hindered  in  the 
future.  They  can  then  plan  acquisition,  information,  and  training  programs  to  erode  these 
barriers  and  ensure  a  swift,  smooth  fielding  and  adoption.  User  training  should 
concentrate  on  increasing  Perceived  Utility,  as  this  is  demonstrably  the  most  important 
acceptance  driver  for  ASK  users.  A  utility  centric  view  of  technology  is  consistent  with 
the  findings  of  other  research,  and  explains  the  insignificant  effect  of  ease  of  use  on 
Behavioral  intent50.  It  follows  that  users  would  not  find  an  innovation  useful  simply 
because  it  is  easy  to  use.  Cultivating  both  peer  and  Chain  of  Command  support  for  the 
new  technology  can  create  a  normative  reinforcement  that  encourages  user  acceptance  as 
well.  While  this  may  not  lead  directly  to  acceptance,  it  can  influence  an  individual 
analyst’s  assessment  of  an  innovation’s  usefulness. 


50  Chienting  Lin,  Paul  Hu,  H.  Chen,  &  J.  Schroeder,  Technology  Implementation  Management  in  Law 
Enforcement:  COPLINK  System  Usability  and  User  Acceptance  Evaluations,  Social  Science  Computer 
Review,  Vol.  22,  No.  1,  2004,  p.  27.- 
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B  RECOMMENDATIONS 

The  intention  of  this  research  has  been  to  use  the  findings  to  aid  in  the 
modification  and  management  of  the  current  Asymmetrical  Software  Kit,  as  well  as  the 
design  and  implementation  of  future  software  packages  of  this  nature. 

1  System  Training  (TNG): 

Training,  according  to  the  results  of  the  survey,  is  the  number  one  discriminator 
for  intent  to  use  (BI).  Military  Intelligence  (MI)  soldiers  who  received  their  training 
through  the  Mobile  Training  Team  or  On  the  Job  Training  displayed  a  lower  intent  to  use, 
a  lower  PEOU,  a  lower  PU,  and  had  a  much  lower  perception  of  the  value  of  the  training 
they  received  on  the  system.  This  stands  in  stark  contrast  to  the  Special  Forces  Soldiers 
(MOS  18F  and  180A),  most  of  whom  received  their  training  through  the  Special  Forces 
Intelligence  Sergeants  Course  (SFISC).  Their  responses  were  the  antithesis  of  the 
responses  from  MI  soldiers.  While  both  of  them  must  use  the  system  to  communicate 
and  share  intelligence  at  the  Tactical  level,  they  do  not  receive  the  same  training.  The 
SFISC  is  a  12  week  course  that  uses  the  ASK  the  entire  time.  The  POI  focuses  on  using 
the  system  in  all  aspects  of  the  18F’s  job,  and  leads  to  a  capstone  project.51  The  capstone 
is  a  culmination  of  their  instruction  on  the  ASK.  Teams  analyze  immense  amounts  of 
real-world  intelligence  data  and  draw  conclusions  about  active  military  conflicts.  Some 
of  these  projects  have  discovered  groundbreaking  information  and  assisted  the  GWOT 
greatly. 

The  disparity  between  the  training  the  18F’s  receive  in  the  SFISC  and  the  training 
the  MI  soldiers  receive  is  alarming,  and  may  be  the  single  largest  driver  in  the  disparity 
between  their  perceptions  of  the  system  and  the  adoption  rates  of  the  ASK.  The  disparity 
doesn’t  have  to  exist,  and  much  of  that  could  be  solved  through  similar,  consistent,  and 
cooperative  training  on  the  system.  The  current  SFISC  proposal  to  fix  this  problem  is  to 
have  contractors  teach  their  POI  to  MI  soldiers  and  replicate  the  18F  course  at  another 


51  SFISC  18F  Program  of  Instruction  (POI)  given  to  the  author  during  interviews  at  Ft.  Bragg,  NC. 
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location.  On  the  MI  side  of  the  house,  the  USASOC  G2  has  a  project  in  concept  stages 
called  the  ARSOF  Military  Intelligence  Assignment  Oriented  Training  (AOT)  Initiative. 
This  AOT  is  designed  to  provide  incoming  MI  soldiers  an  in-depth  training  package  to 
prepare  them  for  the  unique  demands  of  the  Army  SOF  environment.52 

The  recommendation  of  this  author  is  that  the  proposals  be  combined.  The  SFISC 
would  change  to  the  Special  Forces  Intelligence  Course,  or  SFIC.  This  training  would 
include  all  MI  MOS’s  with  the  18F’s  to  standardize  unify  the  training  they  receive  on  the 
ASK.  It  is,  after  all,  the  system  designed  to  help  them  analyze  and  communicate  Tactical 
intelligence.  USASOC  trains  communications  equipment  in  a  collaborative  environment. 
Signal  soldiers  sit  in  the  same  classroom  as  the  Special  Forces  Communication  Sergeants 
(18E’s)  and  work  together  to  establish  communications.  I  would  only  be  natural  that  the 
Special  Forces  Intelligence  Sergeants  should  attend  the  same  training  as  the  Military 
Intelligence  Soldiers  that  support  them.  The  POI  for  the  MI  Soldiers  could  cover  the  new 
and  unique  expectations  they  will  face,  and  the  courses  on  the  ASK  could  be  a  combined 
effort.  MI  soldiers  and  the  18F’s  could  collaborate  on  the  capstone  project,  and  use  the 
ASK  to  piece  together  real  world  intelligence  from  ongoing  conflicts  and  produce 
realistic  products  as  they  would  do  in  combat.  This  would  serve  many  ends.  They  MI 
and  Special  Forces  MOS  Soldiers  would  train  together  on  the  system  they  use  on  a  daily 
basis  to  build  their  confidence  in  the  system.  Their  collaboration  would  also  boost  their 
confidence  in  each  others  abilities,  and  promote  trust  in  the  soldiers  they  will  have  to 
work  with  on  a  daily  basis.  The  relationships  forged  in  these  joint  training  environments 
are  proven  to  be  very  strong  bonds. 

2  Command  Influence  -  Subjective  Norm  (SN): 

In  “Best  Practices  in  Counterinsurgency,”  Kalev  Sepp  wrote  that,  “Intelligence 
operations  that  help  detect  terrorist  insurgents  for  arrest  and  prosecution  are  the  single 
most  important  practice  to  protect  a  population  from  threats  to  its  security.”53  This  is  a 

52  Emails  and  conversations  with  LTC  Battle,  USASOC  deputy  G-2  and  SGM  Coffman,  USASOC 
G2  SGM  about  the  AOT  initiative. 

53  Kalev  Sepp,  Best  Practices  in  Counterinsurgency,  Military  Review,  May-June  2005,  pp.  8-12. 
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change  in  the  norm  for  many  classically  trained  Intelligence  professionals.  According  to 
Les  Grau,  a  respected  Counter-Insurgency  theorist,  Intelligence  necessary  for  fighting 
insurgents  is  a  radical  shift,  and  a  lot  of  change  to  swallow.  “The  military  intelligence 
effort  devoted  to  combating... insurgency  has  little  in  common  with  conventional 
intelligence  operations  in  support  of  conventional  maneuver  war.  Intelligence  preparation 
of  the  battlefield,  order  of  battle,  templating  forces,  signals  intelligence;  measurement  and 
signature  intelligence;  and  electronic  intelligence  take  different  forms  or  are  not 
applicable.  The  S2  or  G2  has  a  different  type  of  war  and  needs  to  take  a  different 
approach  to  dealing  with  it,  much  as  the  U.S.  approach  to  peacekeeping  evolved  during 
the  past  decade.”54  Any  change  this  large  demands  command  involvement  at  a  level 
even  more  than  would  be  normally  necessary  to  simply  shepherd  an  innovation.  This 
change  isn’t  just  about  a  new  system,  it’s  about  a  whole  new  way  of  analyzing 
Intelligence. 

When  infusing  a  new  innovation  into  an  organization,  military  units  need  to  plan 
and  execute  software  fielding  like  a  corporation.  The  current  fielding  model  often  ends  at 
the  purchase  of  software,  neglecting  several  key  steps  to  gain  user  buy-in.  According  to 
John  P.  Kotter,  organizational  transformations,  including  adoption  of  new  software,  can 
be  fraught  with  peril.  He  studied  eight  factors  that  controlled  success  or  failure  of 
adaptation:  Not  enough  sense  of  urgency,  Not  Creating  a  Powerful  Enough  Coalition, 
Lacking  a  Vision,  Under  communicating  the  Vision  by  a  Factor  of  Ten,  Not  Removing 
Obstacles  to  the  Vision,  Not  Systematically  Planning  for  and  Creating  Short-Term  Wins, 
Declaring  Victory  Too  Soon,  and  Not  Anchoring  Changes  in  the  Organization’s 
Culture.55  This  data  has  shown  that  Chain  of  Command  as  having  no  correlation 
whatsoever  to  users  acceptance  decisions.  It  is  obvious  in  this  instance  that  the  Chains  of 
Command  have  under-communicated  their  vision,  declared  victory  after  fielding,  and  not 
institutionalized  (anchored)  this  innovation  in  their  organizational  culture. 

54  Lester  Grau,  Something  Old,  Something  New:  Guerrillas,  Terrorists,  and  Intelligence  Analysis, 
Military  Review,  July-August  2004,  accessed  on  10  December  2007  from  Small  Wars  Journal  website  at 
http://www.smallwarsjournal.com/documents/grau.pdf. 

55  John  P.  Kotter,  “Leading  Change:  Why  Transformation  Efforts  Fail,”  Harvard  Business  Review, 
March- April  1995,  pp.  59-67. 
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The  results  of  measuring  the  Subjective  Norm  (SN)  in  this  study  show  that 
leadership  in  USASOC  is  not  a  strong  influence  in  Soldiers  decisions  to  adopt  the  ASK. 
Peers,  immediate  supervisors,  and  the  entire  Chain  of  Command  should  be  very 
important  social  influences  on  an  operator,  but  they  are  not  involved  enough  in  this 
transformation  of  Intelligence.  Educating  the  leadership  about  the  benefits  of  the  ASK, 
and  helping  them  understand  the  importance  of  using  the  system  is  paramount.  The 
Chain  of  Command  must  also  mandate  usage  across  the  board,  and  remove  old  systems 
to  assist  in  adoption.  Hartwick  and  Barki  found  that  mandatory  usage  was  significantly 
related  to  Intention  to  Use.  Mandatory  users  give  greater  weight  to  the  opinions  of 
others,  using  the  system  more  frequently  when  their  leadership  thinks  that  usage  is 
appropriate.56  In  a  military  environment,  a  change  like  this  should  be  simple  enough. 
The  chain  of  command  can  mandate  usage  of  the  ASK  and  have  the  legacy  systems 
removed  from  the  computers.  In  fact,  many  of  the  survey  respondents  and  soldiers 
interviewed  at  Ft.  Bragg  admitted  they  still  used  Falconview,  an  older  and  simpler  GIS, 
because  their  peers  still  used  it  and  the  Chain  of  Command  tacitly  supported  this  choice. 
Wilson  stated  that  “Tasks  that  are  familiar,  easy,  professionally  rewarded,  or  well  adapted 
to  the  circumstances  in  which  the  operators  find  themselves  will  be  preferred  because 
performing  them  is  less  costly  than  undertaking  tasks  that  are  new,  difficult,  or 
professionally  unrewarded  or  that  place  the  operator  in  conflict  with  his  or  her  own 
environment.”57  By  demanding  the  Soldiers  produce  their  Intelligence  products  with  the 
ASK  and  removing  competing  systems  like  Falconview,  commanders  can  greatly  affect 
adoption  rates  in  their  units.  These  changes  have  to  be  swift  and  visible  to  all  users,  lest 
users  continue  to  ignore  the  innovations  around  them. 

One  of  the  suggestions  brought  out  in  the  interviews  was  for  the  commanders  to 
institute  a  weekly  briefing.  It  was  suggested  that  the  program  be  called  “Fox  Friday”  (for 
the  18  F  Intel  Sergeant  MOS),  and  that  the  18F’s  and  the  MI  Soldiers  of  the  S2  (Intel) 

56  J.  Hartwick  and  H.  Barki,  Examining  the  Role  of  User  Participation  in  Information  System  Use, 
Management  Science,  Vol  40,  (4),  April  1994,  pp.  440-465. 

57  J.  Wilson,  Bureaucracy:  What  Government  Agencies  Do  and  Why  They  Do  It,  New  York,  Basic 
Books,  1989,  p.  231. 
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collaborate  on  a  weekly  briefing  to  the  entire  Battalion  Chain  of  Command.  A  routine 
like  this  will  do  several  things.  First,  it  will  demonstrate  command  knowledge  of  the 
system,  and  expectations  of  system  use.  Second,  a  program  like  this  will  go  a  long  ways 
towards  positively  anchoring  system  usage  in  the  culture  of  the  unit  and  ensuring  that 
new  users  are  brought  into  the  fold  as  they  enter  the  unit.  Having  the  SOF  and  MI  MOS 
soldiers  collaborate  in  a  weekly  preparation  and  briefing  will  cause  them  to  work  together 
towards  a  common  goal,  and  strengthen  trust  and  working  relationships  to  stand  the 
stresses  of  combat. 

3  System  Availability  (AV): 

The  mean  score  for  questions  of  system  availability  belied  an  overall  negative 
impression  of  the  number  and  type  of  systems  available  to  run  the  software.  Many  of  the 
respondents  to  the  survey  noted  that  there  were  concerns  with  both  system  availability, 
and  system  suitability.  Soldiers  expressed  concerns  that  the  number  of  systems  their 
units  were  issued  were  insufficient  to  cover  each  and  every  user,  and  that  many  of  the 
systems  the  ASK  was  loaded  on  were  not  fast  enough  to  run  complex,  memory  intensive 
programs  like  ArcGIS. 

During  interviews  at  Ft.  Bragg,  one  Warrant  Officer  admitted  that  he  had  upgrade 
the  laptop  by  purchasing  enough  RAM  and  a  fast  enough  processor  for  his  1 8F  to  use  the 
system.  His  detachment  CF-50  (Panasonic  Toughbook  Laptop)  was  two  years  old,  had 
survived  three  rotations  to  Southern  Afghanistan  and  was  no  longer  performing  at  the 
level  necessary  to  run  memory  intensive  programs  like  ArcGIS  without  crashing. 
Another  senior  NCO  noted  that,  during  the  SFISC,  the  computers  they  use  are  state  of  the 
art  desktops  maintained  in  a  lab  environment.  They  are  hard-wired  together  and  pull  data 
and  imagery  off  a  dedicated  server.  Once  he  returned  to  his  unit,  his  ASK  laptop  could 
not  match  the  performance  of  the  computers  in  the  lab,  and  that  he  was  greatly  frustrated 
by  this.  One  respondent  even  suggested  using  a  COTS  gaming  computer  like  an 
Alienware™,  with  an  extremely  fast  processor  and  extra  RAM  to  run  ArcGIS. 
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Issues  like  these  have  serious  impacts  on  user  acceptance,  and  can  be  easily 
remedied  by  updating  Basis  of  Issue  to  reflect  real  needs,  proper  acquisition  (with  end 
user  input)  and  property  accountability. 

C.  CONCURRENT  RESEARCH: 

This  project  is  only  the  first  in  a  series  being  executed  by  the  Naval  Postgraduate 
School’s  Common  Operational  Research  Environment  (CORE)  Lab  to  investigate  further 
improvements  that  might  be  made  to  the  ASK.  MAJ  Derek  McClain  will  be  continuing 
this  research,  and  attempting  to  identify  standardized  data  models  and  database  formats 
that  will  facilitate  communication  and  speed  analysis  across  USASOC. 

D.  SUGGESTIONS  FOR  FURTHER  STUDY 

As  the  problems  addressed  in  this  research  are  corrected  and  the  ASK  continues 
to  evolve,  the  program  should  be  evaluated  again  to  examine  new  and  improved  features. 
TAM  is  an  extremely  effective  tool  for  this  type  of  ongoing  evaluation.  Future 
evaluations  and  scores  can  be  conducted,  and  system  evolution  can  be  contrasted  against 
this  research  as  a  baseline.  IT  is  ever  changing  and  evolving,  and  any  lessons  learned 
from  this  analysis  require  that  further  evaluation  take  place  as  the  ASK  goes  through 
subsequent  updates  and  improvements.  Also,  a  user’s  beliefs  and  attitude  towards  an 
innovation  are  likely  to  change  and  evolve  over  time.58  Therefore,  it  will  be  important  to 
measure  both  initial  acceptance,  as  well  as  continued  acceptance  of  the  system.  New 
information  technologies  are  continually  being  introduced  to  the  community  that  could 
benefit  from  acceptance  theory  testing.  Outside  the  realm  of  evaluating  a  new  bit  of 
software  for  the  command,  the  ASK  concept  presents  a  plethora  of  IT  management  issues 
that  could  be  studied  and  researched. 


58  E.  Karahanna,  D.  Straub,  &  N.  Chervany,  Information  Technology  Adoption  Across  Time:  A 
Cross-Sectional  Comparison  of  Pre- Adoption  and  Post- Adoption  Beliefs,  MIS  Quarterly,  23(2)  1999,  pp. 
183-213. 
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E. 


CONCLUSIONS 


Design  and  use  of  advanced  analytic  technologies  is  paramount  to  enhancing  the 
capabilities  of  USASOC  Intelligence  assets  in  the  GWOT.  To  ensure  these  technologies 
are  adopted  and  utilized  to  the  fullest  extent,  the  adopting  units  must  manage  user 
acceptance  in  the  design,  testing,  fielding,  and  adoption.  Motivated  by  the  lack  of 
previous  research  in  this  area,  this  study  developed  and  empirically  tested  a  model  for 
explaining  tactical-level  users’  acceptance  of  the  Asymmetrical  Software  Kit.  Overall, 
the  model  demonstrated  a  good  fit  with  the  data  collected,  and  exhibited  satisfactory 
explanatory  power  to  the  technology  acceptance  decisions  observed  among  Tactical  level 
users  of  the  ASK. 

The  ASK  was  designed  to  enhance  the  analytic  capabilities  of  the  Special 
Operations  Soldiers  at  the  Tactical  level  of  the  GWOT.  In  light  of  the  continuing 
fighting,  and  new  and  developing  fronts  in  this  “long  war,”  effective  adoption  strategies 
for  needed  innovations  are  especially  crucial.  Complex  innovations  that  are  rapidly 
fielded  during  wartime  need  special  attention  paid  to  their  implementation  lest  they  wind 
up  expensive  trinkets  locked  in  a  safe  and  forgotten. 

Intelligence  analysis  is  an  extremely  complex  series  of  tasks  that  have  to  be 
continuously  coordinated.  Recent  advances  in  software  have  produced  several  tools  that 
have  excellent  potential  for  application  to  the  analysis  and  production  process.  The 
programs  in  the  ASK  have  many  tools  and  functions  that  make  analysis  easier  and  more 
effective  for  Intelligence  professionals.  To  make  full  use  of  this  concept,  we  must  train 
ASK  users  together,  to  a  common  standard  of  interoperability,  give  them  computers  that 
are  fast  enough,  and  educate  the  Chain  of  Command  to  make  use  of  the  ASK  a  priority 
for  Intelligence  analysis  at  the  Tactical  level. 
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project  proposal  to  the  NPS  IRB. 


Sincerely* 


Lt  Brent  Olde,  Ph.D. 

Chair 

NPS  Institutional  Review  Board 
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APPENDIX  B  SURVEY  QUESTIONS 


1.  Introduction 


Naval  Postgraduate  School 
Informed  Consent  Statement 

Introduction:  You  are  invited  to  participate  in  a  study  entitled  "Assessing  the  acceptance  and  functional  value  of  the 
Asymmetric  Software  Kit  (ASK)  at  the  tactical  level"  being  conducted  by  the  Department  of  Defense  Analysis, 

Procedures:  Please  be  frank  and  honest  about  your  experiences  with  the  ASK.  This  survey  should  take  about  30  minutes  of 
your  time.  It  is  completely  anonymous,  and  no  identifying  data  will  be  stored  with  your  response. 

Risks  and  Benefits:  This  project  does  not  involve  greater  then  minimal  risk  and  involves  no  known  reasonably  foreseeable 
risks  or  hazards  greater  than  those  encountered  in  everyday  life.  Benefits  of  the  survey  include  a  better  understanding  by 
the  researcher  of  opinions  and  attitudes  within  the  tactical  level  SOF  community  regarding  the  Asymmetric  Software  Kit. 

Compensation:  There  will  be  no  tangible  compensation  provided  for  taking  this  survey.  The  results  of  the  survey  will  be 
available  from  the  USASOC  G2  and  in  thesis  form  from  the  Naval  Postgraduate  School. 

Confidentiality  &  Privacy  Act:  All  records  of  this  study  will  be  kept  private.  No  information  will  be  publicly  accessible  which  could 
identify  participants.  This  survey  will  be  identified  only  as  a  code  number  on  all  research  forms/data  bases.  Any  records  of 
participation  will  be  maintained  by  for  three  years,  after  which  they  will  be  destroyed. 

Voluntary  Nature  of  the  Study.  Participation  is  strictly  voluntary,  if  you  agree  to  participate,  you  are  free  to  withdraw  at  any 
time  without  prejudice. 

Points  of  Contact.  If  there  are  any  questions  or  comments  regarding  this  project  upon  the  completion  of  participation, 
contact  the  Principal  Investigator,  Richard  Bergin,  831-656-3112,  rdbergin@nps.edu,  or  researcher  MAJ  Peter  H.  Hopewell, 
910-489-4517,  phhopewe@nps.edu.  Any  other  questions  or  concerns  may  be  addressed  to  the  IRB  Chair,  LT  Brent  Olde, 
656-3807,  baolde@nps.edu. 

Statement  of  Consent.  I  have  been  provided  with  a  full  explanation  of  the  purpose,  procedures,  and  duration  of  participation 
in  this  research  project.  I  understand  how  my  identification  will  be  safeguarded  and  agree  to  participate  in  this  study.  I 
understand  that  by  agreeing  to  participate  in  this  research  I  do  not  waive  any  of  my  legal  rights.  This  survey  is  anonymous 
and  does  not  require  any  identifying  personal  data.  Clicking  the  NEXT  button  below  Is  your  consent. 


2.  Demographic  Information 


1*  Gender 

BMale 
Female 

2.  MOS 

18F 
180A 
35D 
96B 
I  96D 
98C 
98  G 

_ 35  IB 

Other  (please  specify) 
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3.  Rank 

QjuniorQES  Qe6  0E7  0E8  OW01  QCW2  0CW3  0CW4  O01  O02  O03  O04 

Enlisted 

E1-E4 


4.  Age 

Q18-22  Q23-25  Q2 


03 


O36'4 


O40 


5.  Time  in  Service 

o  0-2  years  03-5  years  O-  years  O9-11  years  O12-15  years  Ql6-18  years  O19  +  years 

6.  Education 

(^)geD  Qhs  Diploma  Qsome  College  (^Associate  (^)bs/BA  (^Postgraduate 

Degree  degree 

7.  Time/  Experience  using  ASK 

(^)l-2  months  (^)3-6  months  (^7-12  months  (^)l3-18  months  (^)l9-24  months  (^)2+  Years 


8.  Type  of  ASK  training  you  received 

{^)l\lew  Equipment  Fielding/  (^)l8F  SFISC 
Mobile  Training  Team 
(pother  (please  specify) 


(^)0n  the  Job  Training  (OJT)  (^)l\lo 


9.  Current  general  computer  usage 


Personal 

never 

O 

About  once  a 

month 

O 

About  once  a 

week 

O 

About  once  or 
twice  a  day 

O 

Three  to  five 

times  a  day 

O 

More  than  five 
times  a  day 

O 

Work 

O 

O 

O 

O 

O 

O 

10*  Current  ASK  Component  Use 

About  once  a 

Mever 

month 

Orion  Magic 

About  once  a 

week 

O 

About  once  or 
twice  a  day 

O 

Three  to  five 
times  a  day 

O 

More  than  five 
times  a  day 

O 

ArcGIS 

O 

O 

O 

O 

O 

O 

Analyst's  notebook 

O 

O 

O 

O 

O 

O 

3.  Technology  Acceptance  Modeling  Survey 
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11.  This  survey  is  conducted  using  a  7-point  Likert  scale  to  state  how  much  you 
agree  or  disagree  with  a  statement.  Please  read  each  question  carefully  (some 
are  negatives)  and  reply  accordingly.  Please  evaluate  the  following  statement 
for  practice: 


The  summer  humidity  in  Fayetteville,  NC  doesn't  bother  me  at  all. 


Strongly 

Agree  with 
the 

Slightly 

Neither 

Agree  ncr 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

The  summer  humidity 
in  Fayetteville,  NC 
doesn't  bother  me  at 

all. 

Comment: 

o 

O 

O 

O 

o 

O 

o 

o 

1 

_ 1 

12.  Using  the  ASK  would  improve  my  job  performance 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

Using  the  ASK  would 

O 

O 

O 

O 

O 

O 

O 

o 

improve  my  job 
performance 
Comments: 


13. 1  could  easily  use  any  of  the  functions  in  the  ASK  on  my  own 


Strongly 

Agree  with 

the 

statement 

Slightly 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

I  could  easily  use  any 

of  the  functions  In  the 

ASK  on  my  own 

Comment: 

o 

o 

o 

O 

o 

o 

o 

o 
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14.  It  is  frustrating  to  use  the  ASK  because  the  computer  I  have  access  to  is 
NOT  fast  enough 


Strongly 
Agree  with 

tiie 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

It  is  frustrating  to  use 

the  ASK  because  the 

O 

O 

O 

o 

o 

O 

o 

o 

computer  I  have  access 
to  is  MOT  fast  enough 
Comment: 

I  ~1 


15.  ASK's  interfaces  make  it  easy  for  me  to  remember  how  to  perform  various 
kinds  tasks 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

ASK's  interfaces  make  it  {^J 

easy  for  me  to 

o 

O 

o 

0 

0 

o 

O 

remember  how  to 
perform  various  kinds 
tasks 
Comment: 


16.  Availability  of  computers  for  accessing  the  ASK  is  going  to  be  a  problem 


Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

Availability  of  computers 
for  accessing  the  ASK  is 
going  to  be  a  problem 

Comment: 

o 

O 

o 

o 

o 

o 

o 

O 

1 

17.  My  supervisor  thinks  I  should  NOT  use  the  ASK 

Strongly 
Agree  with 
the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

My  supervisor  thinks  I 
should  NOT  use  the  ASK 

0 

o 

o 

o 

0 

O 

o 

o 

Comment: 
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18.  I  do  NOT  feel  comfortable  using  the  ASK  on  my  own 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

I  do  NOT  feel 

comfortable  using  the 
ASK  on  my  own 

Comment: 

o 

o 

O 

O 

o 

o 

o 

o 

1 

1 

19.  Whenever 

possible, 

I  would 

use  the  ASK  for 

my  job  tasks 

Strongly 
Agree  with 

the 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

Whenever  possible,  I 
would  use  the  ASK  for 
my  Job  tasks 

Comment: 

O 

o 

O 

O 

o 

o 

o 

O 

1  ~ 

20.  There  are  no  barriers  to  my 

using  the  ASK  for  my  job  tasks 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

There  are  no  barriers  to 

my  using  the  ASK  for 

O 

O 

O 

O 

O 

O 

o 

my  job  tasks 

Comment: 

r 

1 

21.  ASK  addresses  my  job  related  needs 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

ASK  addresses  my  job 

related  needs 

o 

O 

O 

o 

o 

O 

o 

o 

Comment: 

1 

1 
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22.  People  who  use  the  ASK  do  NOT  tend  to  get  recognized  by  their  Chain  of 
Command 


People  who  use  the  ASK 
do  NOT  tend  to  get 
recognized  by  their 
Chain  of  Command 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

O 

O 

o 

o 

o 

O 

o 

o 

23. 1  would  have  no  difficulty  explaining  why  my  use  of  the  ASK  may  (or  may 
not)  improve  my  job  performance 


Strongly 
Agree  with 
the 

state  me  nt 


Agree  with 

the 

statement 


Slightly 
Agree  with 
the 

statement 


I  would  have  no 
difficulty  explaining  why 
my  use  of  the  ASK  may 
(or  may  not)  improve 
my  job  performance 
Comment: 

I 


o 


o  o 


Neither 
Agree  nor 
Disagree 
with  the 
statement 

O 


Slightly 
Disagree 
with  the 
statement 

O 


Disagree 
with  the 
statement 

o 


Strongly 
Disagree 
with  the 
statement 

O 


N/A 


O 


24.  ASK  allows  me  to  accomplish  more  work  than  would  be  otherwise  possible 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

ASK  allows  me  to 

accomplish  more  work 

o 

O 

o 

o 

o 

o 

o 

o 

than  would  be  otherwise 

possible 

Comment: 
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25.  It  would  be  undesirable  for  ASK  to  link  to  the  relevant  external  databases 
(e.g.  those  in  other  intelligence  agencies  or  other  service  Intel  components) 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

It  would  be  undesirable 

for  ASK  to  link  to  the 

O 

o 

o 

o 

O 

o 

o 

relevant  external 
databases  (e,gr  those 
In  other  Intelligence 
agencies  or  other 
service  Intel 
components) 
Comment: 


26.  Using  ASK  would  be  incompatible  with  the  way  I  work 


Using  ASK  would  be 
incompatible  with  the 
way  I  work 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

O 

O 

O 

O 

O 

O 

Strongly 
Disagree 
with  the 
statement 


N/A 


27.  The  various  functions  of  the  ASK  programs  (Orion  Magic,  Analyst's 
Notebook,  and  ArcGIS)  are  easily  accessible 


Strongly 
Agree  with 

the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 
Agree  ngr 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 
with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

The  various  functions  of 

the  ASK  programs 

O 

O 

o 

o 

O 

o 

o 

(Orion  Magic,  Analyst  s 
Notebook,  and  ArcGIS) 
are  easily  accessible 
Comment: 
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28.  Using  the  ASK  saves  time 

Strongly 
Agree  with 

the 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 
Disagree 
with  the 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

N/A 

statement 

statement 

statement 

statement 

Using  the  ASK  saves 

time 

O 

O 

O 

O 

O 

O 

O 

o 

Comment: 

1 

_ 1 

29.  My  colleagues  (fellow  Analysts/  18F's)  do  NOT  think 

I  should  use  the  ASK 

Strongly 

Agree  with 
the 

statement 

Slightly 

Neither 

Agree  nor 
Disagree 

with  the 

Slightly 

Disagree 
with  the 

statement 

Strongly 

Agree  with 
the 

statement 

Agree  with 
the 

statement 

Disagree 
with  the 

statement 

Disagree 
with  the 

statement 

N/A 

statement 

My  colleagues  (fellow 
Analysts/1 8  Fs)  do  NOT 
think  J  should  use  the 

ASK 

Comment: 

O 

O 

O 

o 

o 

O 

o 

o 

1 

_ 1 

30. 1  believe  I  could  communicate  to  others  the  benefit  of  using  the  ASK 

Strongly 
Agree  with 

the 

Agree  with 

the 

Statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

1  believe  I  could 
communicate  to  others 
the  benefit  of  using  the 

ASK 

Comment: 

O 

o 

O 

O 

O 

o 

O 

O 

r  . 

l 

31.  ASK  does  NOT  address  the  critical  aspects  of  my  job 

Strongly 
Agree  with 
the 

Agree  with 
the 

statement 

Slightly 
Agree  with 
the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 
Disagree 
with  the 

Disagree 
with  the 

statement 

Strongly 
Disagree 
with  the 

N/A 

statement 

statement 

statement 

statement 

ASK  does  WOT  address 

the  critical  aspects  of 
my  job 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

1 
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32. 1  would  use  ASK  more  if  I  knew  my  boss  wanted  me  to 


I  would  use  ASK  more  If 
I  knew  my  boss  wanted 
me  to 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

O 

O 

O 

O 

O 

O 

O 

o 

33.  The  computer  I  have  is  slow  and  makes  using  the  ASK  difficult 


Strongly 
Agree  with 

the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

The  computer  I  have  is 
slow  and  makes  using 
the  ASK  difficult 

Comment: 

o 

o 

o 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

O 

o 

o 

o 

o 

34-  When  I  have  access  to  ASK,  I  would  use  it  as  often  as  needed 


Strongly 
Agree  with 
the 

statement 


Agree  with 
the 

statement 


Slightly 
Agree  with 
the 

statement 


Neither 
Agree  nor 
Disagree 
with  the 
statement 


Slightly 
Disagree 
with  the 
statement 


Disagree 
with  the 
statement 


Strongly 
Disagree 
with  the 
statement 


When  I  have  access  to 
ASKj  I  would  use  it  as 
often  as  needed 
Comment: 


o 


o  o  o 


o  o  o 


N/A 


o 


35.  Accessing  relevant  Intelligence  databases  through  the  ASK  is  NOT 
important 


Strongly 
Agree  with 
the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 
with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

Accessing  relevant 
Intelligence  databases 

O 

O 

O 

o 

0 

O 

o 

o 

through  the  ASK  is  NOT 

important 

Comment: 


63 


36.  Using  ASK  would  NOT  make  it  easier  to  do  my  job 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

Using  ASK  would  NOT 
make  it  easier  to  do  my 
job 

Comment: 

O 

O 

O 

O 

O 

O 

O 

37.  I  would  be  able  to  use  the  ASK  effectively  even  if  there  is  no  one  around  to 
show  me  how  to  do  it 


Strongly 
Agree  with 
the 

statement 


I  would  be  able  to  use 
the  ASK  effectively  even 
if  there  is  no  one 
around  to  show  me  how 
to  do  it 
Comment: 


o 


Agree  with 
the 

statement 


Slightly 
Agree  with 
the 

statement 


o  o 


Neither 
Agree  nor 
Disagree 
with  the 
statement 

o 


Slightly 
Disagree 
with  the 
statement 

o 


Disagree 
with  the 
statement 

o 


Strongly 
Disagree 
with  the 
statement 

o 


N/A 


o 


38.  In  my  job,  using  the  ASK  is  important 


Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

In  my  job,  using  the 
ASK  is  important 

Comment: 

O 

O 

O 

O 

O 

O 

O 

O 

1 

_ 1 

39.  The  quality 

of  the  products  I 

am  able  produce  with  the  ASK 

is  high 

Strongly 
Agree  with 
the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

N/A 

The  quality  of  the 
products  I  am  able 

o 

O 

O 

O 

o 

o 

o 

o 

produce  with  the  ASK  is 
high 

Comment: 


64 


40. 1  have  difficulty  finding  a  computer  to  use  ASK  when  I  need  it 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

I  have  difficulty  finding 
a  computer  to  use  ASK 

when  I  need  It 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

1 

□ 

41.  It  would  be  easy  for 

me  to  become  skillful  in 

using  the  ASK 

Strongly 
Agree  with 

the 

Agree  with 

Slightly 
Agree  with 

the 

Neither 
Agree  nor 

Slightly 

Disagree 

with  the 

Disagree 

Strongly 

Disagree 

with  the 

the 

Disagree 
with  the 

statement 

with  the 

N/A 

statement 

statement 

statement 

stat  em  e  nt 

statement 

It  would  be  easy  for  me 
to  become  skillful  In 
using  the  ASK 

Comment: 

o 

O 

o 

O 

o 

o 

o 

o 

1 

_ 1 

42.  It  is  easy  for  me  to  find  the  information  and 

commands  I  need  on  the  ASK 

screens 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

It  is  easy  for  me  to  find 

the  information  and 

commands  I  need  on 

the  ASK  screens 

Comment: 

O 

O 

O 

O 

O 

O 

O 

O 

1 

_ 1 

43.  Interacting 

with  the  ASK  does  NOT  require  a  lot  of  mental  effort 

Strongly 
Agree  with 
the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

N/A 

Interacting  with  the  ASK 
does  MOT  require  a  lot 

of  mental  effort 

o 

O 

O 

O 

o 

o 

o 

o 

Comment: 

1 

.! 
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44-  The  ASK  constantly  produces  substandard  Intel  Products  for  my  effort 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

The  ASK  constantly 
produces  substandard 
Intel  Products  for  my 

effort 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

1 

_ 1 

45,  ASK  is  rigid  and  inflexible  to 

interact  with 

Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 
with  the 

Strongly 

Agree  with 
the 

statement 

Agree  with 
the 

statement 

Disagree 
with  the 

Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

statement 

ASK  is  rigid  and 
inflexible  to  interact  with 

O 

O 

O 

o 

o 

o 

o 

o 

Comment: 

1 

_ 1 

46.  My  interaction  with 

the  ASK 

is  clear  and  understandable 

Strongly 
Agree  with 

the 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

My  interaction  with  the 
ASK  is  clear  and 

understandable 

Comment: 

O 

o 

O 

O 

O 

o 

O 

O 

1 

47. 1  would  be  more  inclined  to  use  the 

ASK  if  my  co  workers  (other  Analysts  or 

ISF's)  were  already  using  it 

Strongly 
Agree  with 
the 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Disagree 
with  the 

statement 

Strongly 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

I  would  be  more 

inclined  to  use  the  ASK 

if  my  co  workers  (other 
Analysts  or  l8Frs)  were 
already  using  St 

Comment: 

O 

O 

O 

o 

0 

O 

o 

o 

J  - 

— 1 
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48.  Overall,  it  is  a  bad  idea  to  use  the  ASK  in  my  job 


Overall,  St  Is  a  bad  Idea 
to  use  the  ASK  in  my 
job 

Comment: 


Strongly 
Agree  with 
the 

statement 


Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

O 

O 

O 

O 

O 

O 

o 

49.  People  who 


People  who  Influence 
my  work  think  I  should 
use  the  ASK 
Comment: 


influence  my  work  think  I  should  use  the  ASK 


Strongly 
Agree  with 

the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

o 

o 

o 

O 

o 

o 

o 

o 

50-  People  who  evaluate  my  job  performance  think  I  should  NOT  use  the  ASK 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

People  who  evaluate  my 
job  performance  think  I 
should  NOT  use  the  ASK 

Comment: 

O 

o 

O 

O 

O 

o 

O 

O 

51.  Overall,  I  find  the  ASK  to  be  useful  to  my  job 

Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

Overall,  I  find  the  ASK 
to  be  useful  to  my  job 

Comment: 

o 

O 

o 

O 

o 

O 

o 

o 
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52. 1  find  the  user  interfaces  in  the  ASK  easy  to  learn 


I  And  the  user 
interfaces  in  the  ASK 
easy  to  learn 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

O 

O 

O 

O 

O 

O 

O 

o 

53.  In  my  job,  use  of  the  ASK  is  relevant 


In  my  Job,  use  of  the 
ASK  is  relevant 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

o 

o 

o 

O 

o 

o 

o 

o 

54.  Using  the  ASK  would  fit  into  my  work  style 


Using  the  ASK  would  fit 
into  my  work  style 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

O 

O 

O 

o 

O 

O 

Strongly 
Disagree 
with  the 
statement 


N/A 


55.  There  are  NOT  enough  computers/ software/hardware  for  everyone  to 
use  ASK 


Strongly 
Agree  with 
the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

There  are  NOT  enough 
computers/soft  ware/ha  rd  ware 

O 

O 

o 

O 

O 

O 

O 

for  everyone  to  use  ASK 
Comment: 


68 


56.  My  immediate  supervisor  thinks  I  should  NOT  use  the  ASK 

Strongly 
Agree  with 

the 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 
Disagree 
with  the 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

N/A 

statement 

statement 

statement 

statement 

My  Immediate 
supervisor  thinks  I 

should  NOT  use  the  ASK 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

□ 

□ 

57.  In  general. 

I  am  NOT  happy 

with  the  products  produced  with  the  ASK 

Strongly 
Agree  with 

the 

statement 

Agree  with 

Slightly 
Agree  with 

the 

statement 

Neither 
Agree  nor 

Slightly 

Disagree 

with  the 

statement 

Disagree 

Strongly 

Disagree 

with  the 

statement 

the 

statement 

Disagree 
with  the 

with  the 

statement 

N/A 

statement 

In  general,  I  am  MOT 
happy  with  the  products 
produced  with  the  ASK 

Comment: 

o 

o 

o 

O 

o 

o 

o 

o 

1 

_ 1 

58.  The  people  I  work  with  think  that  I  should  use  the  ASK 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

The  people  I  work  with 
think  that  I  should  use 

the  ASK 

Comment: 

O 

o 

O 

O 

O 

o 

O 

O 

1 

□ 

59.  The  benefits  of  using  the  ASK  are  NOT  apparent  to  me 

Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

The  benefits  of  using 

the  ASK  are  NOT 

apparent  to  me 

Comment: 

o 

O 

o 

O 

o 

O 

o 

o 

1 

□ 
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60.  Using  the  ASK  would  enhance  my  effectiveness  at  work 


Using  the  ASK  would 
enhance  my 
effectiveness  at  work 
Comment: 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

O 

O 

O 

O 

O 

O 

O 

o 

J 


61.  It  is  easy  to  navigate  between  the  different  search  queries,  output 
products,  and  function  screens  on  the  ASK 


Strongly 
Agree  with 

the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 
with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

It  Is  easy  to  navigate 
between  the  different 
search  queries,  output 
products ,  and  function 
screens  on  the  ASK 

o 

o 

o 

o 

o 

o 

o 

o 

Comment: 

1 

HI 

62.  The  ASK  is  incompatible  with  the  way  I  work 

Strongly 

Agree  with 

the 

statement 

Slightly 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

The  ASK  is  incompatible 
with  the  way  I  work 

Comment: 

O 

O 

O 

O 

O 

O 

O 

O 

1 

_ 1 

63.  The  individuals  with 

whom  I 

work  routinely  think  I  should  use  the  ASK 

Strongly 
Agree  with 
the 

Agree  with 

the 

statement 

Slightly 
Agree  with 
the 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

N/A 

statement 

statement 

statement 

statement 

The  Individuals  with 

whom  I  work  routinely 

think  1  should  use  the 

ASK 

Comment: 

o 

O 

O 

O 

o 

o 

o 

o 
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64.  The  ASK  runs  slowly  on  the  computer  I  have  access  to  because  the 
machine  It  too  old/  worn  out 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

The  ASK  runs  slowly  on 
the  computer  I  have 

O 

o 

o 

o 

O 

o 

o 

access  to  because  the 
machine  it  too  old/  worn 
out 

Comment: 


65.  If  I  need  someone's  help  using  the  ASK,  I  can  NOT  easily  get  it 


Strongly 
Agree  with 
the 

statement 


IF  I  need  someone's 
help  using  the  ASK,  I 
can  NOT  easily  get  it 
Comment: 


Agree  with 
the 

statement 


Slightly 
Agree  with 
the 

statement 


Neither 
Agree  nor 
Disagree 
with  the 
statement 


Slightly 
Disagree 
with  the 
statement 


Disagree 
with  the 
statement 


Strongly 
Disagree 
with  the 
statement 


N/A 


oooooooo 


66. 1  am  NOT  able  to  use  the  ASK  in  my  job  when  I  want  to 


Strongly 
Agree  with 
the 

statement 


I  am  NOT  able  to  use 
the  ASK  In  my  Job  when 
I  want  to 
Comment: 


o 


Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

O 

O 

O 

O 

O 

O 

O 

67,  The  ASK  training  sessions  and  tutorials  that  I  received  are  NOT  helpful 


Strongly 
Agree  with 
the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 
with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

N/A 

The  ASK  training 

sessions  and  tutorials 

O 

O 

O 

o 

O 

O 

O 

o 

that  I  received  are  NOT 

helpful 

Comment: 


71 


68. 1  have  received  adequate  training  on  the  ASK 


Strongly 
Agree  with 

the 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Disagree 
with  the 

statement 

Strongly 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

I  have  received 

adequate  training  on 

the  ASK 

Comment: 

o 

O 

O 

o 

o 

O 

O 

o 

1 

69.  Using  the  ASK  would  be  unpleasant 

Strongly 

Agree  with 
the 

Slightly 

Neither 

Agree  ncr 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

Agree  with 

the 

Disagree 

with  the 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

s  ta  tern  e  nt 

statement 

Using  the  ASK  would  be 
unpleasant 

Comment: 

o 

o 

o 

o 

o 

O 

o 

o 

1 

_ 1 

70.  Using  the  ASK  in  my  job  would  NOT 

increase  my  productivity 

Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

Using  the  ASK  in  my  job 
would  NOT  increase  my 
productivity 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

1 

ZD 

71.  Learning  to  operate  the  ASK  would  be  easy  for  me 

Strongly 
Agree  with 

the 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

N/A 

statement 

statement 

statement 

statement 

Learning  to  operate  the 
ASK  would  be  easy  for 

me 

Comment: 

O 

O 

O 

O 

O 

o 

O 

o 

1 

_ 1 

72 


72. 1  have  no  problem  telling  others  about  the  satisfactory  results  of  using  the 

ASK 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

I  have  no  problem 
telling  others  about  the 
satisfactory  results  of 
using  the  ASK 

Comment: 

O 

O 

o 

o 

o 

O 

o 

o 

1 

□ 

73. 1  have  problems  getting  help  when  using  the  ASK 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

Statement 

with  the 

statement 

statement 

I  have  problems  getting 
help  when  using  the 

ASK 

O 

o 

O 

O 

O 

o 

O 

o 

Comment: 

1 

□ 

74.  The  ASK  allows  me  to  accomplish  more  tasks 

quickly 

Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

Strongly 

Agree  with 
the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

The  ASK  allows  me  to 

accomplish  more  tasks 
quickly 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

1 

_ 1 

75.  Using  the  ASK  would  NOT  be  beneficial  to  my  work 

Strongly 
Agree  with 
the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

N/A 

Using  the  ASK  would 

NOT  be  beneficial  to  my 

work 

o 

O 

O 

O 

o 

o 

o 

o 

Comment: 

1 

□ 

73 


76.  Linking  the  ASK  to  Databases  resident  in  other  Intel  Agencies  is  desirable 

Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

Strongly 
Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

Linking  the  ASK  to 

Databases  resident  in 

other  Intel  Agencies  is 

desirable 

Comment: 

O 

O 

O 

O 

O 

O 

O 

o 

1 

_ 1 

77.  Linking  ASK  to  Databases  resident  in  other  USASOC  units  is  NOT  important 

Strongly 

Agree  with 

the 

Slightly 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 
with  the 

Strongly 

Agree  with 
the 

statement 

Agree  with 
the 

Statement 

Disagree 
with  the 

Disagree 
with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

statement 

Linking  ASK  to 

Databases  resident  in 

other  USASOC  units  is 

NOT  important 

Comment: 

O 

O 

O 

o 

o 

O 

o 

o 

1 

_ 1 

78.  Overall,  I  find  ASK  easy  to  use 

Strongly 

Agree  with 

the 

statement 

Slightly 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  tiie 

statement 

Strongly 

Agree  with 

the 

Agree  with 

the 

Disagree 

with  the 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

Overall,  I  find  ASK  easy 

to  use 

Comment: 

O 

O 

O 

O 

O 

O 

O 

O 

r  . 

79.  Given  that  I  have  access  to  ASK,  I  intend  to  use  it  to 

accomplish  my 

duties 

as  often  as  needed 

Strongly 
Agree  with 
the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 
the 

statement 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 

with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

N/A 

Given  that  I  have 

access  to  ASK,  I  intend 
to  use  it  to  accomplish 
my  duties  as  often  as 
needed 

Comment: 

o 

O 

O 

O 

o 

o 

o 

o 

1 

1 
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30. 1  find  it  easy  to  get  the  ASK  to  do  what  I  want  it  to  do 


Strongly 
Agree  with 

the 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

Disagree 
with  the 

statement 

Strongly 

Disagree 

with  the 

N/A 

statement 

statement 

statement 

statement 

I  find  it  easy  to  get  the 

ASK  to  do  what  I  want  It 

to  do 

Comment: 

o 

O 

O 

o 

o 

O 

O 

o 

1 

81-  There  are  NOT  enough  computers  in 

my  unit  for  me 

to  use  ASK  when 

I 

need  to 

Strongly 
Agree  with 

Agree  with 

the 

statement 

Slightly 
Agree  with 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

Disagree 

with  the 

statement 

Strongly 

Disagree 

N/A 

the 

the 

with  the 

with  the 

statement 

statement 

statement 

statement 

There  are  NOT  enough 
computers  in  my  unit 

O 

O 

O 

o 

O 

O 

O 

o 

for  me  to  use  ASK  when 

I  need  to 

Comment: 


82.  Using  the  ASK  reduces  the  time  I  spend  completing  my  job-related  tasks 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

N/A 

Using  the  ASK  reduces 
the  time  I  spend 
completing  my  job- 

related  tasks 

Comment: 

1 

O 

ZD 

O 

O 

O 

o 

O 

o 

o 
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83. 1  am  NOT  pleased  with  the  quality  of  the  Intel  products  I  can  generate  with 
the  ASK 


I  am  NOT  pleased  with 
the  quality  of  the  Intel 
products  I  can  generate 
with  the  ASK 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

Strongly 

Disagree 

with  the 

statement 

N/A 

statement 

with  the 

statement 

statement 

o 

O 

o 

o 

o 

O 

o 

o 

Comment: 

I  ~1 


84. 1  feel  comfortable  with  the  level  of  in-house  support  available  for  my  using 
the  ASK 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

I  feel  comfortable  with 

the  level  of  in-house 

0 

o 

O 

o 

0 

0 

o 

support  available  for  my 
using  the  ASK 
Comment: 


N/A 


o 


85. 1  do  NOT  have  access  to  the  resources  I  would  need  to  use  the  ASK 
successfully  in  my  job 


Strongly 

Agree  with 

the 

statement 

Slightly 

Neither 
Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Strongly 

Agree  with 
the 

statement 

Agree  with 
the 

statement 

Disagree 
with  the 

statement 

Disagree 
with  the 

statement 

N/A 

I  do  NOT  have  access  to 

the  resources  I  would 

need  to  use  the  ASK 

successfully  In  my  job 

Comment: 

o 

o 

o 

O 

o 

o 

o 

o 

76 


86,  The  computer  I  have  access  to  has  adequate  processing  power  to  run  the 
ASK 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 
with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

The  computer  I  have 
access  to  has  adequate 
processing  power  to  run 

o 

O 

O 

o 

o 

O 

o 

the  ASK 
Comment: 


N/A 


O 


87,  To  the  extent  possible,  I  intend  to  use  the  ASK  in  my  job 


Strongly 
Agree  with 

the 

statement 

Agree  with 

the 

statement 

Slightly 
Agree  with 

the 

statement 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

Strongly 

Disagree 

with  the 

statement 

To  the  extent  possible, 

I  intend  to  use  the  A5K 

in  my  job 

Comment: 

O 

O 

O 

O 

O 

o 

O 

1 

_ 1 

N/A 


o 


88.  The  organization  of  information  on  the  ASK  screens 


Strongly 
Agree  with 
the 

statement 


Agree  with 

the 

statement 


Slightly 
Agree  with 
the 

statement 


Neither 
Agree  nor 
Disagree 
with  the 
statement 


Slightly 
Disagree 
with  the 
statement 


The  organization  of 
information  on  the  ASK 
screens  is  clear  and 
intuitive 
Comment: 


o 


o  o  o 


o 


is  clear  and  intuitive 


Disagree 
with  the 
statement 


Strongly 
Disagree 
with  the 
statement 


N/A 


o  o  o 


89. 1  have  the  opportunities  and  knowledge  I  need  to  use  the  ASK  in  my  job 


Strongly 
Agree  with 
the 

statement 

Agree  with 
the 

statement 

Slightly 
Agree  with 

the 

statement 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 
Disagree 
with  the 

statement 

Disagree 
with  the 

statement 

Strongly 
Disagree 
with  the 

statement 

1  have  the  opportunities 
and  knowledge  1  need 
to  use  the  ASK  in  my 
job 

Comment: 

O 

O 

O 

o 

o 

o 

o 
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90.  The  layout  for  the  various  ASK  program  screens  (Orion  Magic,  Analyst's 
Notebook,  and  ArcGIS)  are  NOT  clear  and  consistent 


Strongly 

Agree  with 
the 

statement 

Slightly 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 
with  the 

statement 

Strongly 

Agree  with 
the 

statement 

Agree  with 
the 

statement 

Disagree 
with  the 

statement 

Disagree 
with  the 

statement 

N/A 

The  layout  for  the 
various  ASK  program 

O 

O 

O 

o 

O 

o 

O 

o 

screens  (Orion  Magic,, 
Analyst's  Notebook,  and 
ArcGIS)  are  NOT  clear 
and  consistent 
Comment: 


91. 1  am  NOT  getting  the  training  1  need  to  be  able  to  use  the  ASK  effectively 


Strongly 

Agree  with 

the 

statement 

Slightly 

Neither 

Agree  nor 
Disagree 

with  the 

statement 

Slightly 

Disagree 

with  tiie 

statement 

Strongly 

Agree  with 

the 

statement 

Agree  with 

the 

statement 

Disagree 

with  the 

statement 

Disagree 

with  the 

statement 

N/A 

I  am  NOT  getting  the 

O 

O 

O 

O 

O 

O 

O 

training  I  need  to  be 
able  to  use  the  ASK 
effectively 
Comment: 


92.  What  features  of  the  ASK  do  you  like  the  most?  Why? 


93.  What  features  of  the  ASK  do  you  like  the  least?  Why? 


94.  What  suggestions  do  you  have  for  future  improvements  to  the  ASK 
concept? 


TTiank  You  very  much  for  your  time.  If  you  have  any  questions  about  the  project,  or  would  like  a  copy  of  my  thesis  when  it  is 
complete,  please  contact  me  at  phhopewe@nps.edu. 
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INITIAL  DISTRIBUTION  LIST 


1 .  Defense  Technical  Information  Center 
Ft.  Belvoir,  Virginia 

2.  Dudley  Knox  Library 
Naval  Postgraduate  School 
Monterey,  California 
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